-- Logs begin at Mon 2022-06-06 11:21:48 UTC, end at Mon 2022-06-06 12:27:12 UTC. -- Jun 06 11:49:01.496714 dv-cinder-825492-20-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 06 11:49:01.506853 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 6 11:49:01 2022] *** Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: nodename: dv-cinder-825492-20-iscsi Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: machine: x86_64 Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: clock source: unix Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: pcre jit disabled Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: detected number of CPU cores: 4 Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: current working directory: / Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: detected binary path: /usr/bin/uwsgi-core Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: your processes number limit is 47680 Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: your memory page size is 4096 bytes Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: detected max file descriptor number: 2048 Jun 06 11:49:01.507445 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: lock engine: pthread robust mutexes Jun 06 11:49:01.508723 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: thunder lock: enabled Jun 06 11:49:01.508723 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 06 11:49:01.508723 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 06 11:49:01.553068 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: Python main interpreter initialized at 0x55943b5b19c0 Jun 06 11:49:01.553068 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: python threads support enabled Jun 06 11:49:01.553068 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: your server socket listen backlog is limited to 100 connections Jun 06 11:49:01.553068 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: your mercy for graceful operations on workers is 90 seconds Jun 06 11:49:01.553396 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: mapped 403077 bytes (393 KB) for 2 cores Jun 06 11:49:01.553492 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: *** Operational MODE: preforking *** Jun 06 11:49:01.553763 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: *** uWSGI is running in multiple interpreter mode *** Jun 06 11:49:01.553763 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: spawned uWSGI master process (pid: 80136) Jun 06 11:49:01.553958 dv-cinder-825492-20-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 06 11:49:01.554666 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: spawned uWSGI worker 1 (pid: 80146, cores: 1) Jun 06 11:49:01.554875 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: spawned uWSGI worker 2 (pid: 80147, cores: 1) Jun 06 11:49:01.556739 dv-cinder-825492-20-iscsi devstack@keystone.service[80136]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 06 11:49:07.263830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 11:49:07.263830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: from cryptography.utils import int_from_bytes Jun 06 11:49:07.264699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 11:49:07.264699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: from cryptography.utils import int_from_bytes Jun 06 11:49:07.265233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 11:49:07.265233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: from cryptography.utils import int_from_bytes Jun 06 11:49:07.265679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 11:49:07.265679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: from cryptography.utils import int_from_bytes Jun 06 11:49:07.313717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 11:49:07.313889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 11:49:07.314084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] command line args: [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 11:49:07.314359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 11:49:07.314537 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ================================================================================ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 11:49:07.314652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 11:49:07.314808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] admin_token = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.314909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 11:49:07.315073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] command line args: [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 11:49:07.315152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config_dir = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.315259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.315343 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 11:49:07.315441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] config_source = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.315518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ================================================================================ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 11:49:07.315618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] debug = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.315744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] admin_token = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.315832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_dir = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] config_source = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] debug = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.316928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_config_append = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.317938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_dir = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_config_append = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_dir = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.318961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.319987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.320966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.321958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.322111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.322450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.322596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.322688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] publish_errors = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.322779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] publish_errors = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.322877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.323994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] standard_threads = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] standard_threads = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_journal = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_json = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.324967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_journal = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_stderr = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_json = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] use_syslog = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_stderr = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] use_syslog = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 11:49:07.325710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.325884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.325972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.326121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.326200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.326974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.327946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.328988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.329935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.330939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.331902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.332989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.333993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.334967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.335886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.336944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.337967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.338870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.339963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.340977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.341932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.342970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343421 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.343890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.344843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.345987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.346962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.347375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.347461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.347612 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.347688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.347843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.347928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.348984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.349995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.350985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.external = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.external = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.351983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.password = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.password = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] auth.token = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] auth.token = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.352891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.353995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.354979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.355999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.356994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.357172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.357249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.357412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.357490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.357757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.357834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.358996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.359933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.360958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.361114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.361191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.361345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.361430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.361590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.361835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.362069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.362303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.363121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.363392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.363653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.363924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.364946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365612 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.365928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.366943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.367940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368327 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.368877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.369998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.370971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.371975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.372906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.373074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.373676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.373838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.373955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.374918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.375999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.376971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.377987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.378941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.379912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380135 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.380989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.381926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.382964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.383913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.384862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.385963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.386919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.387958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.388124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.388201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.388366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.388452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.388610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.388688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.389418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.389663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.389739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.389801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.389986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.390826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.391976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.392957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.393910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.394959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.395980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.396916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.397961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398358 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.398922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.399878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.400901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.401984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.402935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.403945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 11:49:07.404968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 11:49:07.405087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80147) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 11:49:07.433761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.433970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.434058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.434203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.434293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.434425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.443359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.443488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.473134 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.473357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.473614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.473847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.528715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.528890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.577827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.578002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.579356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.579594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.644875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.645948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.646030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.670019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.670414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.670649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.670964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.671991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.683024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.683125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.695849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.696948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.697030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.697161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80147) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.697243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.697383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.697597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 11:49:07.715113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.715302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.715386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 11:49:07.715513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 11:49:07.715644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.715845 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 11:49:07.715927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 11:49:07.716175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716320 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.716923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.718109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.728460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.728732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.728994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.729251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.729336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.729527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.729794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.730074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.737570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.738585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.740853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.741134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.741413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.741705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.741902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.742011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.742171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.742256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.742456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.742540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.742722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.742813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.743803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.744070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.744339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.744603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.770820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.771845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.772461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.772736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.773474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.773754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.778343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.778620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.778928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.779262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.779708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.780244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.786324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.786605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.787334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.787620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.791083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.791367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.791653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.791934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.792128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.792211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.792418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.792501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.792700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.792782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.792983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.793067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.793260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.793532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.793799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.794177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.813220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.813492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.814190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.814468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.818111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.819150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.821871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.822173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.822936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.823231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.827277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.828408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.830893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.831996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.834501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.834797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.835104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.835387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.835656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.835740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.835943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.836027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.836225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.836512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.836791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.837070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.855303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.855586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.855847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.856104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.856576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.856860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.857139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.857410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.864555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.866107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.867803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.868105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.869465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.869744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.871254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.871544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.871824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.872101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.872378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.872832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.873123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.873454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.873733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.874015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.885770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.886050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.886333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.886614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.886934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.887216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.887389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.887498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.887672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.887955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.888237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.888517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.889329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.889617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.906008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.907589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.909275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.910820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.912750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.914310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.914401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.915957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.916045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.916272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.916559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.917593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.917870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.918152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.924760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.926031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.926368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.927675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.929731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.930020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.930304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.930584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.931057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.931349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.931633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.931920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.940281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.940567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.940846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.941588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.941874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.942153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.948832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80147) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.949110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.949384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.949655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.949923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.950159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 11:49:07.950243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.950441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.950533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.950712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.950794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.951910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.952153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.952237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.952420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80147) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.952508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.952694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.952961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.953239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.953511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.953769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 11:49:07.991444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55943b5b19c0 pid: 80147 (default app) Jun 06 11:49:07.992447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55943b5b19c0 pid: 80146 (default app) Jun 06 11:49:07.994803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865474e-2ac2-4908-a232-1b60b6161d9a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:07.995118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865474e-2ac2-4908-a232-1b60b6161d9a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:07.995397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865474e-2ac2-4908-a232-1b60b6161d9a None None] PATH_INFO: `/v3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:08.000233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1/1] 10.222.0.41 () {50 vars in 791 bytes} [Mon Jun 6 11:49:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:16.162582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:16.163156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:16.163156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:16.167415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1/2] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:16.173118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:16.173529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:16.173683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:16.243582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: INFO dbcounter [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] Registered counter for database keystone Jun 06 11:49:16.264782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG oslo_db.sqlalchemy.engines [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80147) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 11:49:16.271060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writer thread running {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 11:49:16.553254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG passlib.handlers.bcrypt [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80147) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 11:49:16.553684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG passlib.handlers.bcrypt [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80147) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 11:49:16.611550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:16.613161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:16.737704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 2/3] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 11:49:16.841892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:16.911821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: INFO dbcounter [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] Registered counter for database keystone Jun 06 11:49:16.931310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG oslo_db.sqlalchemy.engines [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80146) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 11:49:16.938356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writer thread running {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 11:49:17.161807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3717cc-46f1-4f84-befb-a72711b0ccde None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:17.166014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:17.166666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:17.166983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:17.167267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:17.175637 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:17.228821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 11:49:17.229164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fd403-ae60-4338-9517-0af87ebb3ad1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:17.238776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 2/4] 10.222.0.41 () {58 vars in 1324 bytes} [Mon Jun 6 11:49:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 11:49:17.244756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:17.260238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32a660-9024-4b5b-bf02-b2da84484912 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:17.264611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:17.266893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:17.267186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:17.267467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] PATH_INFO: `/v3/projects/a70232cae3264ff0a2f3b7ce99f87508` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:17.268961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] RBAC: Authorizing `identity:get_project(project_id=a70232cae3264ff0a2f3b7ce99f87508)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:17.321580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING keystone.common.rbac_enforcer.enforcer [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 11:49:17.321928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e275f6-f97b-47dc-89b1-b1c49bf0dbba admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:17.328817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 3/5] 10.222.0.41 () {56 vars in 1207 bytes} [Mon Jun 6 11:49:17 2022] GET /identity/v3/projects/a70232cae3264ff0a2f3b7ce99f87508 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:19.532803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b84fb-c289-489b-abd8-a6d16d1d0468 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:19.533396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b84fb-c289-489b-abd8-a6d16d1d0468 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:19.533396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b84fb-c289-489b-abd8-a6d16d1d0468 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:19.537462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 3/6] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:19.542971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18503042-b754-4f4d-9cf5-13110121178e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:19.543399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18503042-b754-4f4d-9cf5-13110121178e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:19.543539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18503042-b754-4f4d-9cf5-13110121178e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:19.563277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-18503042-b754-4f4d-9cf5-13110121178e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:19.564809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-18503042-b754-4f4d-9cf5-13110121178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:19.572560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 4/7] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 11:49:19.595635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b84fb-c289-489b-abd8-a6d16d1d0468 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:19.608192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b84fb-c289-489b-abd8-a6d16d1d0468 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:19.612500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5185dd8-b819-48a6-9051-1bea8dfb6a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:19.613130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185dd8-b819-48a6-9051-1bea8dfb6a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:19.613423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185dd8-b819-48a6-9051-1bea8dfb6a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:19.613708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185dd8-b819-48a6-9051-1bea8dfb6a1a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:19.621438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5185dd8-b819-48a6-9051-1bea8dfb6a1a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:19.623247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5185dd8-b819-48a6-9051-1bea8dfb6a1a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:19.632283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 4/8] 10.222.0.41 () {58 vars in 1324 bytes} [Mon Jun 6 11:49:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 11:49:19.637384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18503042-b754-4f4d-9cf5-13110121178e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:19.646781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18503042-b754-4f4d-9cf5-13110121178e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:19.651125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c841dae-75b1-4f40-bd2f-874452a1a815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:19.651976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c841dae-75b1-4f40-bd2f-874452a1a815 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:19.652263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c841dae-75b1-4f40-bd2f-874452a1a815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:19.652542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c841dae-75b1-4f40-bd2f-874452a1a815 admin admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:19.653921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c841dae-75b1-4f40-bd2f-874452a1a815 admin admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:19.655794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c841dae-75b1-4f40-bd2f-874452a1a815 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:19.657933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 5/9] 10.222.0.41 () {56 vars in 1198 bytes} [Mon Jun 6 11:49:19 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:21.968608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf735b8-8343-49f7-9ba9-688f158c3c1c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:21.969721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf735b8-8343-49f7-9ba9-688f158c3c1c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:21.970021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf735b8-8343-49f7-9ba9-688f158c3c1c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:21.971418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 5/10] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:21.976931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540223a4-d695-4a95-b8a6-ca92b161160d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:21.977371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540223a4-d695-4a95-b8a6-ca92b161160d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:21.977724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540223a4-d695-4a95-b8a6-ca92b161160d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:21.994355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-540223a4-d695-4a95-b8a6-ca92b161160d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:21.995569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-540223a4-d695-4a95-b8a6-ca92b161160d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:22.076269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 6/11] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:22.100674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf735b8-8343-49f7-9ba9-688f158c3c1c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.101487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b613b-a370-44aa-9430-e8e4156eb793 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.101790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b613b-a370-44aa-9430-e8e4156eb793 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.102085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b613b-a370-44aa-9430-e8e4156eb793 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.103477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 7/12] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:22.108362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.108659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.108950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.125571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:22.126869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:22.143854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.195568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 8/13] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:22.217503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.245856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf735b8-8343-49f7-9ba9-688f158c3c1c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.279261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.280052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.280336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.280613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.292758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:22.304405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:22.305515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:22.306351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 6/14] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:22.311283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.319187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67651a-0a2a-4847-8ff3-95ce024c3c6e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.345855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.353165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.353819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.354110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.354393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.376472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.414427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4d8de-0199-46e6-b5c7-9b5c14b6f26c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.423959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.425464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.447482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.448153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.448445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.448731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.449482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:22.449798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.451216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.463399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 7/15] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:22.469530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.524766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 9/16] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 11:49:22.530178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.562318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.569071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0f11b-8d0b-45e3-9ddc-70c339f68113 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.583690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.603043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.603920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.604209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.604486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.606084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.607589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.609399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 8/17] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:22.615203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.631895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb22b14-44b7-4c5a-b762-45eea3f1925b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.666203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.667026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.667308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.667594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.677763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.679318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:49:22.699778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:22.700730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:22.701524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:49:22.701524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:49:22.701524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:49:22.701524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:22.701524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 10/18] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:22.707303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.717439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905cc6e-f45f-46cd-9ca8-28a80b42f1cb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.752091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.752884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.753179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.753452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.754705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.756194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.757778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 9/19] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:22.763414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:22.795187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.809317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e41906-bc8c-453f-89e4-7260f44ecfea None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.813740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:22.850849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.851551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.851843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.852130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.852860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:22.853171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.854589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332ff2-5b5b-4af6-a8fd-6e7315c61f64 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.868430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6f3e3-c9c9-4fd5-b8ce-e895037803be None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:22.873260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 11/20] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:22.904320 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:22.905006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:22.905322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:22.905608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:22.906535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.domain.id=default, user.id=b408ca5ee3cd4f8b9e1f61c55e26955b) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:22.906964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:22.909025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5c930-132b-4469-a729-e38f9715e8ee None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:22.918889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 10/21] 10.222.0.41 () {56 vars in 1322 bytes} [Mon Jun 6 11:49:22 2022] GET /identity/v3/role_assignments?user.id=b408ca5ee3cd4f8b9e1f61c55e26955b&scope.domain.id=default&role.id=b4418dc4009641b183fe42e56f1e484e => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:25.529071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58debee-ab7a-4016-ac08-6f0512edfa2a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:25.529579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa562a-d12a-445a-8840-87c55318c040 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:25.529579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa562a-d12a-445a-8840-87c55318c040 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:25.530008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa562a-d12a-445a-8840-87c55318c040 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:25.530633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58debee-ab7a-4016-ac08-6f0512edfa2a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:25.531142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58debee-ab7a-4016-ac08-6f0512edfa2a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:25.531239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 11/22] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:25.533296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 12/23] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:25.538784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:25.539100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:25.539388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:25.550449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:25.551083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:25.551576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:25.561661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG passlib.handlers.bcrypt [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80146) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 11:49:25.562091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG passlib.handlers.bcrypt [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80146) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 11:49:25.587898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:25.589742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:25.609302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:25.612004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:25.747784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 13/24] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:25.776447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 12/25] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:25.800381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:25.813641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:25.864875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:25.978859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:25.986405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c3529-0d59-4a7b-b403-f35f0a04d212 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.034529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194755-b385-4c57-b927-c3e80fcf1a5a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.052771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.053509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.053798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.054079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.076940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:26.078454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:26.101168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.102223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.102511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.102790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:49:26.108350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:26.109157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:26.110129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:49:26.110129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:49:26.110129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:49:26.110129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:26.110129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 14/26] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:26.128234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:26.136118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:26.137203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:26.138122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:26.139030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:26.141926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 13/27] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:26.148886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.292745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:26.307302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4620d-7ad5-48fe-b0bb-c15af75e5ed7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.316476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9da08-1068-4b8a-a507-598099152e27 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.345765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.346828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.347362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.347834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.348752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:26.349350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:26.351238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499561-2a66-4ae3-92be-efed7b0b59d4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:26.358571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.360474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.360914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.361338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.362745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:26.363252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:26.365106 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d5639-8994-4449-beec-1eeb26881939 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:26.365634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 15/28] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:26.371589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c93f-e5c0-4077-8c77-31039bba92b2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.372104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c93f-e5c0-4077-8c77-31039bba92b2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.372604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59c93f-e5c0-4077-8c77-31039bba92b2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.374093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 16/29] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:26.374683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 14/30] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:26.376232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bff97-9f1d-425c-810c-5886f113392b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.376525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bff97-9f1d-425c-810c-5886f113392b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.376811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bff97-9f1d-425c-810c-5886f113392b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.376916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c93f-e5c0-4077-8c77-31039bba92b2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.378461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 15/31] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:26.383524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bff97-9f1d-425c-810c-5886f113392b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.389925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:26.494049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59c93f-e5c0-4077-8c77-31039bba92b2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.536236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bff97-9f1d-425c-810c-5886f113392b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.538752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.540086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.540574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.541067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.542424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:26.545232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:26.580359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e202d3d-3fab-4c6b-af36-88503966df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.581263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e202d3d-3fab-4c6b-af36-88503966df6b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.581560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e202d3d-3fab-4c6b-af36-88503966df6b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.581843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e202d3d-3fab-4c6b-af36-88503966df6b None admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.583537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e202d3d-3fab-4c6b-af36-88503966df6b None admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:26.585052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e202d3d-3fab-4c6b-af36-88503966df6b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:26.586966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 16/32] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:26 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:26.587963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758a08b-de2d-4095-9105-73ee7bb9e473 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.588256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758a08b-de2d-4095-9105-73ee7bb9e473 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.588539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758a08b-de2d-4095-9105-73ee7bb9e473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.607390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5758a08b-de2d-4095-9105-73ee7bb9e473 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:26.608588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5758a08b-de2d-4095-9105-73ee7bb9e473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:26.621411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:26.682118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 17/33] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:26.688297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.688602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.688893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.702291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 17/34] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:49:26 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:49:26.708255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.719892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:26.725022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:26.805651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 18/35] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:26.806391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.822884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:26.891204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87522dd3-8d78-4484-9df1-5dd5a762c7ac None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:26.912341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:26.932253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:26.933077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:26.933367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:26.933649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:26.935021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:26.936635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:26.939054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 18/36] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:26.939864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:26.948558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fcb14-a3c8-4107-b445-99502a120907 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:27.004874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:27.006199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.006199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.006199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.006941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:27.008971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:27.071191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607ca95-598c-4a8a-954e-095cac028b54 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:27.105031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:27.116726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2139b-50ad-4ab2-9a83-946620977792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:27.117408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2139b-50ad-4ab2-9a83-946620977792 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.117701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2139b-50ad-4ab2-9a83-946620977792 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.117987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2139b-50ad-4ab2-9a83-946620977792 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.118748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2139b-50ad-4ab2-9a83-946620977792 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:27.120198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2139b-50ad-4ab2-9a83-946620977792 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:27.135698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 19/37] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 11:49:26 2022] POST /identity/v3/roles => generated 219 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:27.143016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 19/38] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 11:49:26 2022] POST /identity/v3/roles => generated 225 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:27.143189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b0940-da23-42bc-9402-fd78b1ddd5de None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.143189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b0940-da23-42bc-9402-fd78b1ddd5de None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.143189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b0940-da23-42bc-9402-fd78b1ddd5de None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.143396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:27.143461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 20/39] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:27.146265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.146265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.146265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.169945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:27.171788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:27.269698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 21/40] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:27.305828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:27.352028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b3d1b-2ea1-411a-bc2b-856a64f374ff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:27.389562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16528589-ef4d-4ae4-a199-94b08e524349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:27.390425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16528589-ef4d-4ae4-a199-94b08e524349 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.390712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16528589-ef4d-4ae4-a199-94b08e524349 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.391031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16528589-ef4d-4ae4-a199-94b08e524349 None admin] PATH_INFO: `/v3/domains/default/users/b408ca5ee3cd4f8b9e1f61c55e26955b/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.394946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16528589-ef4d-4ae4-a199-94b08e524349 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b408ca5ee3cd4f8b9e1f61c55e26955b, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:27.396446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16528589-ef4d-4ae4-a199-94b08e524349 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:27.401799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:27.416644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 20/41] 10.222.0.41 () {58 vars in 1361 bytes} [Mon Jun 6 11:49:27 2022] PUT /identity/v3/domains/default/users/b408ca5ee3cd4f8b9e1f61c55e26955b/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:27.453587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5f257-5f8c-4657-9e5d-d31d00d13312 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.454710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5f257-5f8c-4657-9e5d-d31d00d13312 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.455347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5f257-5f8c-4657-9e5d-d31d00d13312 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.456924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 21/42] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:27.462922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.463508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.464000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.479385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9d99-d6c9-4a17-8d83-71f85149649c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:27.484235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:27.485916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:27.500102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:27.529101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:27.530066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.531090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.532711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.534788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:27.535965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:27.555145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 22/43] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 11:49:27 2022] POST /identity/v3/roles => generated 223 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:27.581366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 22/44] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:27.606613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:27.725078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909f6b7-04f5-4f17-9baa-d9dd05cc59c1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:27.762340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03150cc-10a2-4c89-a950-515cd47a9447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:27.763559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03150cc-10a2-4c89-a950-515cd47a9447 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.764067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03150cc-10a2-4c89-a950-515cd47a9447 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.764557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03150cc-10a2-4c89-a950-515cd47a9447 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.765780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03150cc-10a2-4c89-a950-515cd47a9447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:27.767742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03150cc-10a2-4c89-a950-515cd47a9447 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:27.769557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 23/45] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:27.775059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:27.883407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2e02f-6688-4a59-89df-9344e70b569f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:27.909169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:27.917985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a0937-345d-4edf-99f6-f40b60995b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:27.918686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0937-345d-4edf-99f6-f40b60995b74 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:27.919023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0937-345d-4edf-99f6-f40b60995b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:27.919324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a0937-345d-4edf-99f6-f40b60995b74 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:27.920061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a0937-345d-4edf-99f6-f40b60995b74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:27.921528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a0937-345d-4edf-99f6-f40b60995b74 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:27.957556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 23/46] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:49:27 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:29.759492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce83cc8-f5f2-493e-bbb9-5dbfa8462d88 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:29.760008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce83cc8-f5f2-493e-bbb9-5dbfa8462d88 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:29.760093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce83cc8-f5f2-493e-bbb9-5dbfa8462d88 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:29.761432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 24/47] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:29.766805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:29.767356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:29.767488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:29.784732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:29.785931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:29.852037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 24/48] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:29.875294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce83cc8-f5f2-493e-bbb9-5dbfa8462d88 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:29.926496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:29.984263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce83cc8-f5f2-493e-bbb9-5dbfa8462d88 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.021733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.022544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.022833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.023151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.035936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:30.044501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:30.045305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:30.046087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 25/49] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:30.052095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:30.150284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bb78b-1958-4d19-8c56-914f13bf34c1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.160588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:30.186512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.187212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.187504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.187788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.188593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:30.189458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:30.190876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2c26-5225-4d39-8a16-8f5d82c0d300 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:30.199252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 25/50] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:30.204571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:30.239981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb8cd5-aa80-4bae-b6c5-f5a30484203a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.240278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb8cd5-aa80-4bae-b6c5-f5a30484203a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.240567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb8cd5-aa80-4bae-b6c5-f5a30484203a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.241933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 26/51] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:30.246828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.247150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.247436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.264564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:30.265749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:30.305281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c855f63-460b-4f5e-8578-c08fc38e8877 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.310891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:30.334975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 27/52] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:30.347658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.348537 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.348831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.349119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.350324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:30.352098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:30.353743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 26/53] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:30 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:30.358360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:30.358930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:30.454515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:30.458509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40051a-908c-4a60-be26-ec55549e8de0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.459657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b86cc-c999-4601-812f-fdffcc63dca4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.493247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.493819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.494042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.494334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.494604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.494692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.494924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.495211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.495832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:30.496393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:30.497280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:30.497867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:30.498950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 28/54] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:30.499517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 27/55] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:30.504895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:30.505024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:30.546781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:30.604369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09aa2f-a2c8-4ef7-93ef-0315f483978a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.606134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba2728-f05d-476a-a499-6c479dc884aa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:30.644634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.645296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.645583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.645866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.646713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.domain.id=default, user.id=b408ca5ee3cd4f8b9e1f61c55e26955b) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:30.647056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:30.647187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b83a-60e8-4a21-af50-4a593355e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:30.647795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b83a-60e8-4a21-af50-4a593355e9ba None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:30.648090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b83a-60e8-4a21-af50-4a593355e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:30.648374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b83a-60e8-4a21-af50-4a593355e9ba None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:30.648560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c989c10-2e56-4aa8-93a8-783c92a17f9e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:30.649092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6b83a-60e8-4a21-af50-4a593355e9ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:30.650546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6b83a-60e8-4a21-af50-4a593355e9ba None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:30.656171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 28/56] 10.222.0.41 () {56 vars in 1322 bytes} [Mon Jun 6 11:49:30 2022] GET /identity/v3/role_assignments?user.id=b408ca5ee3cd4f8b9e1f61c55e26955b&scope.domain.id=default&role.id=b4418dc4009641b183fe42e56f1e484e => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:30.677077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 29/57] 10.222.0.41 () {60 vars in 1140 bytes} [Mon Jun 6 11:49:30 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:49:32.943025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d5bb6-7a0f-4e53-baac-91a49f4b4bde None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:32.943569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d5bb6-7a0f-4e53-baac-91a49f4b4bde None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:32.943569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d5bb6-7a0f-4e53-baac-91a49f4b4bde None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:32.944976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 29/58] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:32.950110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:32.950494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:32.950691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:32.968443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:32.969636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:32.983511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:33.036450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 30/59] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:33.060580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d5bb6-7a0f-4e53-baac-91a49f4b4bde None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:33.161176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d5bb6-7a0f-4e53-baac-91a49f4b4bde None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:33.163456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:33.194527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9bc9f-9c97-43ec-a1ac-8a3c8c071b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:33.195286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9bc9f-9c97-43ec-a1ac-8a3c8c071b43 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:33.195579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9bc9f-9c97-43ec-a1ac-8a3c8c071b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:33.195866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9bc9f-9c97-43ec-a1ac-8a3c8c071b43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:33.197036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9bc9f-9c97-43ec-a1ac-8a3c8c071b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:33.198496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9bc9f-9c97-43ec-a1ac-8a3c8c071b43 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:33.200137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 30/60] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:33.205760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:33.302403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91800c-a046-4230-a64d-38f85a7190d1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:33.335751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef929a-5ab3-4b0d-a452-3ce5fdfcd31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:33.336417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef929a-5ab3-4b0d-a452-3ce5fdfcd31f None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:33.336710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef929a-5ab3-4b0d-a452-3ce5fdfcd31f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:33.336996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef929a-5ab3-4b0d-a452-3ce5fdfcd31f None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:33.337699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef929a-5ab3-4b0d-a452-3ce5fdfcd31f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:33.339188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef929a-5ab3-4b0d-a452-3ce5fdfcd31f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:33.364716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 31/61] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:49:33 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:49:36.935975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8e9df-397c-49f0-a9a6-e5754d3f78ea None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:36.936500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8e9df-397c-49f0-a9a6-e5754d3f78ea None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:36.936574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8e9df-397c-49f0-a9a6-e5754d3f78ea None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:36.937993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 31/62] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:36.957773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c676f5-af1d-48ec-8949-3dde1124be27 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:36.958284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c676f5-af1d-48ec-8949-3dde1124be27 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:36.958356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c676f5-af1d-48ec-8949-3dde1124be27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:36.974596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:36.987518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-35c676f5-af1d-48ec-8949-3dde1124be27 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:36.988778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-35c676f5-af1d-48ec-8949-3dde1124be27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:36.992292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82997e9a-eef9-43ca-9759-d8ff2272bb1f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:36.992591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82997e9a-eef9-43ca-9759-d8ff2272bb1f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:36.992932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82997e9a-eef9-43ca-9759-d8ff2272bb1f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:36.994363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 32/63] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:37.000350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.000995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.001614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.039063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:37.040285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:37.094289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 32/64] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:37.096087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47cd18-2c10-4cef-851d-1b323920dc00 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.096557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47cd18-2c10-4cef-851d-1b323920dc00 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.097002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47cd18-2c10-4cef-851d-1b323920dc00 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.098710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 33/65] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:37.109216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.109715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.110178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.129856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:37.132287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:37.154230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 33/66] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:37.155532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.219893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 34/67] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:37.221295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.306765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:37.318801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7ab3-fea2-4769-92f4-820dcfe4a64e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.320985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:37.356030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec39d-1ca0-4dfd-9166-0133a25f6f6a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.357116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.358102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.358995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.359320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.372813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:37.381091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:37.382011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:37.382766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 34/68] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:37.383433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.391458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19a257-b56a-4236-9994-603adc931fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.393062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19a257-b56a-4236-9994-603adc931fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.393356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19a257-b56a-4236-9994-603adc931fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.393643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19a257-b56a-4236-9994-603adc931fa6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.403773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19a257-b56a-4236-9994-603adc931fa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2d19a257-b56a-4236-9994-603adc931fa6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:37.412097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:37.413063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 35/69] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:37.418817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249325f4-d4f5-44d9-b2f4-6757365a6070 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249325f4-d4f5-44d9-b2f4-6757365a6070 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249325f4-d4f5-44d9-b2f4-6757365a6070 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 36/70] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6c0e9-21b5-4c2a-8e15-7b8036f0b149 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6c0e9-21b5-4c2a-8e15-7b8036f0b149 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6c0e9-21b5-4c2a-8e15-7b8036f0b149 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.419839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 37/71] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:37.420200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6c0e9-21b5-4c2a-8e15-7b8036f0b149 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.486899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472d50-9077-4da0-8cdb-b12ef5e3d41b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.519565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.520375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.520668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.520958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.526414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6c0e9-21b5-4c2a-8e15-7b8036f0b149 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.531438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:37.540329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:37.541212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:37.542006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 35/72] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:37.542617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.559355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:37.568601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:37.571348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.572038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.572323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.572604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.573367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:37.573682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:37.575115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed320a-8717-4698-9df3-e90db137eb43 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:37.584395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 38/73] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:37.585688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea66671-fc66-4743-96e6-b83fef12ab48 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.585994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea66671-fc66-4743-96e6-b83fef12ab48 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.586278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea66671-fc66-4743-96e6-b83fef12ab48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.603797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8ea66671-fc66-4743-96e6-b83fef12ab48 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:37.605016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8ea66671-fc66-4743-96e6-b83fef12ab48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:37.651326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4517b5d-fb02-4e08-99c4-3a0fb7f18223 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.671530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 39/74] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:37.672820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.673157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.673501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.687151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.687835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.688127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.688415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.689410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:37.689759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:37.691618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:37.698238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:37.699468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:37.705852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 36/75] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:37.706697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.769145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 40/76] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:37.769925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.791296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:37.807949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba62281-9e5a-4ea9-b605-5b170daac0bf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.821403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:37.841843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.842519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.842817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.843134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.843891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:37.844211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:37.845684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:37.854110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 37/77] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:37.859347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.875464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e18acf-7dbc-4233-8f87-b0962cab9afd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.909187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.910062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.910351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.910632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.926009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:37.927530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:37.929215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 41/78] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:37.929997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:37.959284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3c92d-459b-4f44-a78a-c2f9673542dd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:37.992635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:37.993629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:37.993956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:37.994276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:37.995514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:37.997009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:37.998624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 38/79] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:37.999444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.015901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.033734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3128742-507e-41a0-bfa5-2097f40c5377 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.040924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.067971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.068845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.069130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.069409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.070581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.072070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.073683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 42/80] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:38.074439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.104385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b11d04-7e8d-4680-aa6f-55e9050ce41e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.137758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.138556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.138881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.139162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.149889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:38.158014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:38.158883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:38.159641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 39/81] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:38.160283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.176799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b9b47-044c-4204-8fba-c71dc15bef48 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.209811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.210684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.211017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.211301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.212472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.213930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.215597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 43/82] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:38.216338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.236866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.259496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.262166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56511e7-5377-4fdf-bc57-81d8d9d777be None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.295526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.296175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.296475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.296742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.316590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eb8af-e48f-484d-84fc-10882cc7ba4d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.349839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.350495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.350781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.351351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.352053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.353502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.363684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.365370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.374880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 44/83] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:49:38 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:38.375871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.467282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.490214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.494306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8cf4-c6fb-448e-8555-c570e66ffbe2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.498373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 40/84] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:38.503904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.536351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.537027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.537321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.537609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.620367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.621792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.633593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da54076-395e-4fbc-8c8f-b7c41bdf174a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.676183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.677472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.678215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.678688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.679679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:38.680009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.682286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.692734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 41/85] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:38.693627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.736351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.743451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:38.749377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 45/86] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:38.749992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.819212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc67f3-da01-4e4f-b516-54bcfae40da9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.868020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.868685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.868982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.869272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.883184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134be1f9-135f-4466-9b9e-c51d6620f994 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:38.919159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:38.920032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:38.920322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:38.920606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:38.921826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.924448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.926408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 46/87] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:38.926998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:38.948394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:38.950104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:38.990869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:39.003674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:39.039073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddddb-5203-460f-bf51-a93a0d79a4e8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:39.069526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 42/88] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:39.070342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.075871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:39.076766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:39.077058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:39.077341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:39.078568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:39.081141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:39.083559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 47/89] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:38 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:39.084512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.188279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e3faf-b132-47b7-add6-0aa65d8c44f9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:39.211785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a83763-8682-4214-847b-e9016cfe7ec1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:39.233723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:39.234593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:39.234904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:39.235241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:39.236455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:39.237939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:39.239834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 43/90] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:39.245508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.249414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:39.256679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:39.257400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:39.257689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:39.258022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:39.259017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=350c85611b984015a23d8c9c7772914f) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:39.259341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:39.260898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:39.263206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:39.269635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 48/91] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/role_assignments?user.id=350c85611b984015a23d8c9c7772914f&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:39.270522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.456636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc9f04-845c-422f-9c4f-c7da99cb7fab None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:39.464937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ff561-264a-4cab-9d0c-47252ee67ca4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:39.506854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:39.507685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:39.508081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:39.509771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:39.536673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:39.538069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:39.538361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:39.538647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:39.540141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:39.543800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:39.546873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 44/92] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:39.549352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.623155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:39.637210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:39.638857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:39.695395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:39.759693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a01cd5-47bf-41bb-a59e-b1fa538a76bd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:39.833730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:39.834401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:39.834689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:39.834997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:39.835845 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] RBAC: Adding query filter params (role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=350c85611b984015a23d8c9c7772914f) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:39.836166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:39.837654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:39.849051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 49/93] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:39.849921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.852367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 45/94] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/role_assignments?user.id=350c85611b984015a23d8c9c7772914f&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=36fb427222cd413e808b7d4028e53faf => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:39.865853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:39.992311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aad95-c7fb-4189-bcef-0c102ed36edc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:40.030623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:40.031541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:40.032453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:40.032853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] PATH_INFO: `/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:40.034414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] RBAC: Authorizing `identity:get_project(project_id=bbe76eed4fd44bbca5de26adc977b9b2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:40.036020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:40.037748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 46/95] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:40.043452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:40.050365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:40.065021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:40.077028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0350-83a6-449b-bc38-a03e94cdd0fa None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:40.157568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92f0e-4a82-4dd5-87fd-d0d186fded8c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:40.164604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:40.165302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:40.165599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:40.165885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:40.166755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=b408ca5ee3cd4f8b9e1f61c55e26955b) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:40.167105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:40.168608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f057db-548a-4bdb-a977-91b90c3c4456 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:40.184881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 50/96] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:39 2022] GET /identity/v3/role_assignments?user.id=b408ca5ee3cd4f8b9e1f61c55e26955b&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:40.202830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:40.203608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:40.203938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:40.204459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:40.205540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=bbe76eed4fd44bbca5de26adc977b9b2, user.id=350c85611b984015a23d8c9c7772914f) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:40.205870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:40.207486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484eb4b-b0dc-43fb-8856-3c688a37e5db None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:40.218997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 47/97] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:40 2022] GET /identity/v3/role_assignments?user.id=350c85611b984015a23d8c9c7772914f&scope.project.id=bbe76eed4fd44bbca5de26adc977b9b2&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:41.202208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ecc25-1c95-4f54-b225-459e9307a15e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.202700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ecc25-1c95-4f54-b225-459e9307a15e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.202771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ecc25-1c95-4f54-b225-459e9307a15e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.204670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 51/98] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:41.210018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0c043-e20d-4959-bfd9-5f8af968f4da None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.210885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0c043-e20d-4959-bfd9-5f8af968f4da None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.211376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0c043-e20d-4959-bfd9-5f8af968f4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.235895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3bb0c043-e20d-4959-bfd9-5f8af968f4da None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:41.237447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3bb0c043-e20d-4959-bfd9-5f8af968f4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:41.311287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:41.318171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 48/99] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:41.362977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ecc25-1c95-4f54-b225-459e9307a15e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:41.486872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69089c8-c57a-44bd-ab38-20989ef62d10 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.487224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69089c8-c57a-44bd-ab38-20989ef62d10 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.487467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69089c8-c57a-44bd-ab38-20989ef62d10 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.488857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 49/100] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:41.493716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.493929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ecc25-1c95-4f54-b225-459e9307a15e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:41.494414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.494937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.528603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:41.529665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:41.530477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.530580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:41.530762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.531087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.541926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:41.550551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:41.551584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:49:41.552331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:41.552895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 52/101] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:41.558883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:41.595319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:41.657553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 50/102] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:41.663360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a3eca-6b89-4eb6-a65b-6e8aeb6cbb1a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:41.694669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:41.706862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:41.706862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.706862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.706862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.707245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:41.707245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:41.707245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:41.728874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 53/103] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:41.734496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:41.863024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67234c56-52d2-4c0e-9123-3d3b5090fb63 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:41.893452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:41.908958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:41.909772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:41.910061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:41.910337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:41.911700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:41.913188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:41.914829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 51/104] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:41.924281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:41.944545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:41.976737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102de47-49f2-49c8-b2a4-40c5884ea693 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:42.061155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:42.062056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.062344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.062666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] PATH_INFO: `/v3/users/f91fa44198524e4ea0124a0f05b3d4b1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.086941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] RBAC: Authorizing `identity:get_user(user_id=f91fa44198524e4ea0124a0f05b3d4b1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:42.088443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:42.090493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 54/105] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:41 2022] GET /identity/v3/users/f91fa44198524e4ea0124a0f05b3d4b1 => generated 289 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:42.096622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:42.160882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92017132-2667-4d1d-b96f-839e42aee761 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:42.243553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7adfc-5e74-41fe-945c-bdcbf835e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:42.245013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7adfc-5e74-41fe-945c-bdcbf835e4fe None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.245503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7adfc-5e74-41fe-945c-bdcbf835e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.246033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7adfc-5e74-41fe-945c-bdcbf835e4fe None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.246986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7adfc-5e74-41fe-945c-bdcbf835e4fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:42.248663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7adfc-5e74-41fe-945c-bdcbf835e4fe None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:42.253787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b66de-b979-4c28-b7b3-18d5a0028711 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:42.318138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:42.319101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.319519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.319938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.321163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 52/106] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:49:41 2022] POST /identity/v3/projects => generated 302 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:42.329253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c5733-9663-40e5-97af-bdf642305c04 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.330088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c5733-9663-40e5-97af-bdf642305c04 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.330582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c5733-9663-40e5-97af-bdf642305c04 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.334996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 53/107] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:42.336349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:42.338347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.338898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.339725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.373673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:42.375901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:42.404553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:42.406190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:42.453726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:42.488731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 54/108] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:42.521909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 55/109] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:42.523150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:42.548793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:42.608028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:42.637167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40340d46-94b5-48eb-9d0e-1ffd4b6117b5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:42.675684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:42.676239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224254b0-3ce0-4608-91e7-b710ca45ef8a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:42.676988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.677454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.677937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.690326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:42.698464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:42.699399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:42.700147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:42.702570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 55/110] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:42.708273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:42.715883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:42.716845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.717138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.717424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.719599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:42.721389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:42.723572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 56/111] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:42 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:42.729579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:42.804971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:42.840483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a978c5-453b-45f8-8d6e-60c1cbd971ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:42.853099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:42.878980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:42.879646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:42.879935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:42.880218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:42.881229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] RBAC: Adding query filter params (role.id=881f45605c544dddb6830e6749387c9d, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=f91fa44198524e4ea0124a0f05b3d4b1) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:42.881548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:42.883023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:42.888936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 57/112] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:42 2022] GET /identity/v3/role_assignments?user.id=f91fa44198524e4ea0124a0f05b3d4b1&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=881f45605c544dddb6830e6749387c9d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:42.955933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559e0fe-eb7f-4ca2-b714-ce74a725ba3e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:43.027104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:43.028160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.028653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.029136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.030133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:43.030649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:43.032753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:43.048296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 56/113] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:43.054514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:43.181833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce31ba-9a46-4c84-930b-3fac42aee8d0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:43.227391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14052136-2ed4-4f21-b0ba-24fd163d05d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:43.228276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14052136-2ed4-4f21-b0ba-24fd163d05d7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.228561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14052136-2ed4-4f21-b0ba-24fd163d05d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.228840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14052136-2ed4-4f21-b0ba-24fd163d05d7 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.230059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14052136-2ed4-4f21-b0ba-24fd163d05d7 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:43.231546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14052136-2ed4-4f21-b0ba-24fd163d05d7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:43.233181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 58/114] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:43 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:43.238790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:43.393898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:43.398202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b4aed-4478-43ae-8164-53616409c382 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:43.439305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb336db1-3a63-436c-bdff-d696bb5fc8e1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.439826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb336db1-3a63-436c-bdff-d696bb5fc8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.440040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb336db1-3a63-436c-bdff-d696bb5fc8e1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.441582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 59/115] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:43.447137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.447436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.447740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.447934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:43.448748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.449041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.449442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.454897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:43.456408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:43.458169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 57/116] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:43 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:43.464856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:43.483931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:43.485184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:43.571841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:43.652532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 60/117] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:43.657308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e8c96-a5ed-45fc-b73b-f33a77457bf0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:43.689886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:43.717626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77cd00-d310-426c-af7a-839acd572a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:43.718768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77cd00-d310-426c-af7a-839acd572a45 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.719083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77cd00-d310-426c-af7a-839acd572a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.719362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77cd00-d310-426c-af7a-839acd572a45 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/350c85611b984015a23d8c9c7772914f/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.721480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77cd00-d310-426c-af7a-839acd572a45 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fced2c17e3741e992d8466df4bd6539, user_id=350c85611b984015a23d8c9c7772914f, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:43.723032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77cd00-d310-426c-af7a-839acd572a45 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:43.750960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 58/118] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:43 2022] PUT /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/350c85611b984015a23d8c9c7772914f/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:43.833012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a287e1-9cc5-49ca-9876-93e58e14ecab None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.833888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a287e1-9cc5-49ca-9876-93e58e14ecab None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.834496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a287e1-9cc5-49ca-9876-93e58e14ecab None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.837465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 59/119] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:43.844525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:43.845879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:43.846391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:43.876223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:43.878669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:43.909387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:43.953751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1184-773e-4d0b-b95c-acf8540a12b6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:43.973127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 60/120] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:43.996522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.040227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.041070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.041365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.041648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.065300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:44.081873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:44.082809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:44.084037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 61/121] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:49:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:44.092112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.168332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:44.194661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e93ed5-fc41-41a8-9547-3c8c3ccaf199 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:44.241669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e029a3-3121-47a5-9626-cb1cb1f7726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.242533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e029a3-3121-47a5-9626-cb1cb1f7726c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.242825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e029a3-3121-47a5-9626-cb1cb1f7726c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.243165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e029a3-3121-47a5-9626-cb1cb1f7726c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.258042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e029a3-3121-47a5-9626-cb1cb1f7726c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-00e029a3-3121-47a5-9626-cb1cb1f7726c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:44.266766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:44.267738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:44.268492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:44.269581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 61/122] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:44.270754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017c7af-6742-4db9-85f0-5cff45c9f35d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.270754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017c7af-6742-4db9-85f0-5cff45c9f35d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.270754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017c7af-6742-4db9-85f0-5cff45c9f35d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.272029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 62/123] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:44.276009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017c7af-6742-4db9-85f0-5cff45c9f35d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.300659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:44.321412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fca2c-f391-44c7-a69a-1aa5e2550982 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:44.398169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017c7af-6742-4db9-85f0-5cff45c9f35d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:44.435434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.436073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.436372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.436659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.437502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:44.437884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:44.439356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55afa3-cb61-403c-8475-1b987dc70c1c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:44.439549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.440293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.440634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.440924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.441695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:44.442012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:44.443498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:44.448975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 63/124] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:44.450709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.451032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.451316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.469904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:44.470255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 62/125] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:44.471074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.475453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:44.551467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 64/126] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:44.552114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.597730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:44.653046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:44.665638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefbf15-9dd8-4250-a8f8-63e21413d796 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:44.682329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df7b22-809d-4cde-9f19-6105fb472966 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:44.701941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.703315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.703610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.703895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.706057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:44.708245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:44.711134 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 63/127] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:44.712114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.723199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.724280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.724577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.725118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.726469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:44.728141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:44.729979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 65/128] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:44.730646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.829828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729b3-0d26-47ff-b3a5-bc2be9ade2d2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:44.874451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:44.875523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:44.875973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:44.876264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:44.887636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:44.895775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:44.898211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 64/129] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:44.900367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:44.936426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:44.963298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:44.989121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99539983-6e9f-4599-8784-5f127603e244 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.016977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddce25-3209-4ab0-8951-ba67ae2a3945 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.062824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.064133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.064459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.064955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.065062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.065284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.065574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.065940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.070293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.070293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.070293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 65/130] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:45.071292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:45.071700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.073382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.079834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 66/131] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:44 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:45.080903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:45.178237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180821-a9b9-4a99-840e-5125dc50ac30 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.191214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:45.218024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.219167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.219531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.219893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.220726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:45.221326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.222859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.232564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 66/132] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:45.233632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:45.344898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107df9e-1f1c-4afb-b3c3-80c7709ad76c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.382331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.383958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.384334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.384697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/b408ca5ee3cd4f8b9e1f61c55e26955b/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.387039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fced2c17e3741e992d8466df4bd6539, user_id=b408ca5ee3cd4f8b9e1f61c55e26955b, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.388611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.412837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a46f8-cbd2-4767-8121-b21684fbe938 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.422364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 67/133] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:45 2022] PUT /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/b408ca5ee3cd4f8b9e1f61c55e26955b/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:45.423285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:45.470778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:45.473644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.474574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.475245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.475544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/350c85611b984015a23d8c9c7772914f/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.481074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] RBAC: Authorizing `identity:create_grant(project_id=6fced2c17e3741e992d8466df4bd6539, user_id=350c85611b984015a23d8c9c7772914f, role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.482579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.509339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 67/134] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:45 2022] PUT /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/350c85611b984015a23d8c9c7772914f/roles/36fb427222cd413e808b7d4028e53faf => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:45.653081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90b06c-7f91-48fd-9a79-61e346a5ded0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.727336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.728176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.728470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.728754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.730099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.731648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.733373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 68/135] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:45 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:45.739792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:45.753254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:45.872190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42edf10-f164-4414-9581-9d3d50a0a7fb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:45.909741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0702c73-2f05-4eb5-8367-d2a3d5524c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:45.910686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702c73-2f05-4eb5-8367-d2a3d5524c0d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.911066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702c73-2f05-4eb5-8367-d2a3d5524c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.911393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702c73-2f05-4eb5-8367-d2a3d5524c0d None admin] PATH_INFO: `/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.912754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0702c73-2f05-4eb5-8367-d2a3d5524c0d None admin] RBAC: Authorizing `identity:get_project(project_id=bbe76eed4fd44bbca5de26adc977b9b2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:45.914416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0702c73-2f05-4eb5-8367-d2a3d5524c0d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:45.916287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 68/136] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:45 2022] GET /identity/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:45.922780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:45.977181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926a3fe-dc70-41e8-bf54-9f67a622e5f5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.977548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926a3fe-dc70-41e8-bf54-9f67a622e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.977819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926a3fe-dc70-41e8-bf54-9f67a622e5f5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:45.979429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 69/137] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:45.989598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c3e31-3cd1-4161-93ac-5495079b6ab9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:45.989896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c3e31-3cd1-4161-93ac-5495079b6ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:45.990187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c3e31-3cd1-4161-93ac-5495079b6ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.024318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-353c3e31-3cd1-4161-93ac-5495079b6ab9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:46.025563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-353c3e31-3cd1-4161-93ac-5495079b6ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:46.063478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:46.102038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38132c42-ee9c-4566-8031-6cf6a1a790a6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:46.131889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 70/138] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:46.169897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5eb4e-8c31-4a36-ab36-e95e4ac27985 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:46.170585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5eb4e-8c31-4a36-ab36-e95e4ac27985 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:46.171720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5eb4e-8c31-4a36-ab36-e95e4ac27985 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.174339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 71/139] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:46.176231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5eb4e-8c31-4a36-ab36-e95e4ac27985 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:46.177964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fd2ee-966c-4ce8-a377-0c8c26811d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:46.179168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd2ee-966c-4ce8-a377-0c8c26811d76 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:46.179669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd2ee-966c-4ce8-a377-0c8c26811d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:46.180152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd2ee-966c-4ce8-a377-0c8c26811d76 None admin] PATH_INFO: `/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2/users/350c85611b984015a23d8c9c7772914f/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.183167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:46.184001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fd2ee-966c-4ce8-a377-0c8c26811d76 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe76eed4fd44bbca5de26adc977b9b2, user_id=350c85611b984015a23d8c9c7772914f, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:46.185651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fd2ee-966c-4ce8-a377-0c8c26811d76 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:46.213920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 69/140] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:45 2022] PUT /identity/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2/users/350c85611b984015a23d8c9c7772914f/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:46.215233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:46.215535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:46.215822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.256009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:46.259055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:46.381068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5eb4e-8c31-4a36-ab36-e95e4ac27985 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:46.406166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 70/141] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:46.441238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:46.442248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:46.442594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:46.442947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.444311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:46.445960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:46.447817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 72/142] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:46.453173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:46.464847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:46.584061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:46.626463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:46.650085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18f105-1c5d-4753-91b9-b39dd9908ee3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:46.695978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41995-e26b-4a70-b528-fae4ab3d7c80 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:46.708138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:46.708824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:46.709105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:46.709384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.710101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:46.711600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:46.760870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 73/143] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:49:46 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:46.770300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:46.771207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:46.771499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:46.771788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:46.791374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:46.799711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:46.800671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:46.801424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 71/144] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:46.808906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:46.951756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380034c-be1f-4464-8635-face8d4df5ce None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:46.993667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:47.002642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:47.008002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.008288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.008568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.009344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:47.009658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:47.016537 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59256e4-2511-4f17-9eb4-cfe63e5f21da None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:47.031033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 74/145] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:47.037186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:47.138641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f40c-1835-4f26-8014-2e062bbe35cb None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.139010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f40c-1835-4f26-8014-2e062bbe35cb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.139230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f40c-1835-4f26-8014-2e062bbe35cb None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.140838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 75/146] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:47.146111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.146402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.146683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.175963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:47.177168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:47.217151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94cfc3-fd76-4052-9967-e1a3152b02a0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:47.243470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:47.255298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:47.256234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.256489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.256764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] PATH_INFO: `/v3/users/f91fa44198524e4ea0124a0f05b3d4b1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.258681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] RBAC: Authorizing `identity:get_user(user_id=f91fa44198524e4ea0124a0f05b3d4b1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:47.260453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:47.270652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 72/147] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:47 2022] GET /identity/v3/users/f91fa44198524e4ea0124a0f05b3d4b1 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:47.272087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:47.283935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 76/148] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:47.330673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:47.453309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353cc1-833c-48c3-9b30-3756d12593d9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:47.476801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4d410-2e5b-4b0b-b78e-cbaea2aa9450 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:47.501289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:47.513388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:47.515331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.515958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.516440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.518316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:47.522567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:47.525216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 73/149] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:47 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:47.535382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:47.535886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:47.536205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.536492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.536772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.559550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:47.567968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:47.568822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:47.569651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:47.574009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 77/150] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:47.579939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:47.670270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:47.741448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114cce-96e7-40f7-8f86-ab3d5cd52e8c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:47.791015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebbda2-3122-43a6-a804-601c86793bef None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:47.804307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:47.806014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.806509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.807072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/f91fa44198524e4ea0124a0f05b3d4b1/roles/881f45605c544dddb6830e6749387c9d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.809782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fced2c17e3741e992d8466df4bd6539, user_id=f91fa44198524e4ea0124a0f05b3d4b1, role_id=881f45605c544dddb6830e6749387c9d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:47.812114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:47.833795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 74/151] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:47 2022] PUT /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539/users/f91fa44198524e4ea0124a0f05b3d4b1/roles/881f45605c544dddb6830e6749387c9d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 11:49:47.860699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:47.861407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:47.861698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:47.861982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:47.862756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:47.863168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:47.864636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:47.879663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 78/152] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:47.885360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:48.032765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae8f70-b127-4af2-91e6-8fc2c1680d08 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:48.061911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:48.073918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:48.074827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:48.077798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:48.078077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:48.080002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:48.081951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:48.084072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 75/153] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:47 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:48.089981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:48.157970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:48.231610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4fd76-8181-4380-b716-b4189a2caa98 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:48.280960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:48.282052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:48.282560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:48.283070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:48.367105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:48.368959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:48.421222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:48.479922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 79/154] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:48.487954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:48.626119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815c0d6-b25c-41aa-b75d-0b6ee139d879 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:48.666195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6ae3-11c6-4ffc-925f-d25aa13617aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:48.667100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6ae3-11c6-4ffc-925f-d25aa13617aa None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:48.667389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6ae3-11c6-4ffc-925f-d25aa13617aa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:48.667667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6ae3-11c6-4ffc-925f-d25aa13617aa None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:48.668885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6ae3-11c6-4ffc-925f-d25aa13617aa None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:48.670444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6ae3-11c6-4ffc-925f-d25aa13617aa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:48.672594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 76/155] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:48 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:48.678608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:48.747911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c351de8-042a-42e1-9187-42b0d78b7d38 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:48.748227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c351de8-042a-42e1-9187-42b0d78b7d38 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:48.748481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c351de8-042a-42e1-9187-42b0d78b7d38 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:48.750497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 77/156] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:48.760291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:48.760585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:48.760873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:48.785098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:48.786799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:48.860165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d9d6-64ed-42c6-9932-2b6ee64a76a1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:48.907213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:48.925038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:48.925706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:48.925948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 78/157] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:48.926046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:48.927028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:48.927277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=350c85611b984015a23d8c9c7772914f) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:48.927599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:48.929135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:48.939824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 80/158] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:48 2022] GET /identity/v3/role_assignments?user.id=350c85611b984015a23d8c9c7772914f&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:48.981379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:49.138216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d52aaf-127a-4c12-bdcd-c7f9a66020d3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:49.181555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:49.182699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:49.183282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:49.183769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:49.200699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:49.209398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:49.210258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:49.211052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:49.213099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 79/159] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:49.219178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:49.234545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:49.372219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666ece9-0c64-44c4-bdf1-ec9555c2a5bc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:49.413342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:49.414026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:49.414316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:49.414597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:49.415391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:49.415850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:49.417261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba4a0-0194-4e28-a30c-0925a09028b7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:49.430952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 81/160] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:49.437326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:49.516344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:49.574855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9bb3e-9d4e-449c-b4c4-c03d8c4f8ab9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:49.578375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c11a5-0731-46c2-8fd7-2032452b33d9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:49.579276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c11a5-0731-46c2-8fd7-2032452b33d9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:49.579825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c11a5-0731-46c2-8fd7-2032452b33d9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:49.581591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 82/161] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:49.588510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:49.589110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:49.589587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:49.628784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:49.631370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:49.636406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:49.637574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:49.638056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:49.638528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] PATH_INFO: `/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:49.640832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] RBAC: Authorizing `identity:get_user(user_id=b408ca5ee3cd4f8b9e1f61c55e26955b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:49.643999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:49.646012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 80/162] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:49 2022] GET /identity/v3/users/b408ca5ee3cd4f8b9e1f61c55e26955b => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:49.655994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:49.769237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:49.778177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 83/163] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:49.805516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daa4dd-2286-48c8-9ea4-3322bd853a1c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:49.819934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:49.828425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:49.867882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:49.867882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:49.867882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:49.867882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:49.960713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:49.962199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:50.089013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6d322-f74f-4251-be72-3805229d92d0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:50.175488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:50.176654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.177155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.177627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.191490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 81/164] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:50.196936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:50.202298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:50.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:50.215823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:50.216589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 84/165] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:49:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:50.224279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:50.232818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:50.374517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:50.394594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e7c84-e700-4ee0-97ab-db9e25f07cd0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:50.471255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdd37e-d6eb-46fb-a279-39d0a427b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:50.472370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdd37e-d6eb-46fb-a279-39d0a427b589 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.472794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdd37e-d6eb-46fb-a279-39d0a427b589 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.473222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdd37e-d6eb-46fb-a279-39d0a427b589 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.475710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323f5b-6104-4a0a-9a06-1222ab8d8823 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:50.476603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdd37e-d6eb-46fb-a279-39d0a427b589 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:50.478222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdd37e-d6eb-46fb-a279-39d0a427b589 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:50.480128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 82/166] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:50 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:50.487727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55198c1f-7f6c-48b1-98e7-8df66142f58a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.488209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55198c1f-7f6c-48b1-98e7-8df66142f58a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.488700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55198c1f-7f6c-48b1-98e7-8df66142f58a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.490511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 83/167] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:50.491454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55198c1f-7f6c-48b1-98e7-8df66142f58a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:50.564255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:50.564940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.565412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.565830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.566727 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:50.567198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:50.568750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d163-418f-45e1-ad05-25f0eaa31232 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:50.585996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 85/168] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:49:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:50.587989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.588433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.588853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.627934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:50.629349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:50.679957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:50.720691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55198c1f-7f6c-48b1-98e7-8df66142f58a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:50.780392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 86/169] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:50.781261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:50.787551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:50.788221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.788514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.788849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.789704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=b408ca5ee3cd4f8b9e1f61c55e26955b) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:50.790024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:50.791526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:50.801566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 84/170] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:50 2022] GET /identity/v3/role_assignments?user.id=b408ca5ee3cd4f8b9e1f61c55e26955b&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:50.876790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:50.892565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:50.934519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6d2e3-7728-4c93-83c8-e45513ca00b9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:50.983816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:50.984729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:50.985014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:50.985290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:50.986637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:50.988187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:50.995753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 87/171] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:50 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:50.996731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:51.065770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e299-9cd7-45af-8358-24c82122187a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:51.103725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:51.104816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.105259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.105689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.119165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:51.128141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:51.129264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:51.130417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:51.132300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 85/172] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:51.141648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:51.163439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354f612-06ec-4caf-bb05-8ac197b28eb9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:51.177524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:51.206395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:51.244293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:51.246292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.246589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.246896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.374275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a14c7-6b25-411b-97aa-878c784aaf73 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:51.382492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:51.384026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:51.463690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475888-99b1-40ca-a713-9735909be70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:51.464304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475888-99b1-40ca-a713-9735909be70e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.464590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475888-99b1-40ca-a713-9735909be70e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.464873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475888-99b1-40ca-a713-9735909be70e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.465632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69475888-99b1-40ca-a713-9735909be70e None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:51.465939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69475888-99b1-40ca-a713-9735909be70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:51.467512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69475888-99b1-40ca-a713-9735909be70e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:51.480022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 86/173] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:51.485278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475888-99b1-40ca-a713-9735909be70e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:51.573935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 88/174] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:51.580618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:51.606661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:51.722485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:51.726181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6464aa-fbd6-4eaa-afed-45bd64ee5221 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:51.770107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44e9f-a3ed-4ca6-9ebf-a3c72cb4c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:51.771494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44e9f-a3ed-4ca6-9ebf-a3c72cb4c7cd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.772756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44e9f-a3ed-4ca6-9ebf-a3c72cb4c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.773863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44e9f-a3ed-4ca6-9ebf-a3c72cb4c7cd None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.776423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44e9f-a3ed-4ca6-9ebf-a3c72cb4c7cd None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:51.779060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44e9f-a3ed-4ca6-9ebf-a3c72cb4c7cd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:51.781554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 89/175] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:51 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:51.783030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c556f0-d492-4eee-86eb-563a8ba48362 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.783538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c556f0-d492-4eee-86eb-563a8ba48362 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.784031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c556f0-d492-4eee-86eb-563a8ba48362 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.785458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 90/176] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:51.790200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c556f0-d492-4eee-86eb-563a8ba48362 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:51.800763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475888-99b1-40ca-a713-9735909be70e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:51.889433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:51.900527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec44038-ad6d-4e70-b121-563256c77d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:51.901346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec44038-ad6d-4e70-b121-563256c77d31 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.901632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec44038-ad6d-4e70-b121-563256c77d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.901906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec44038-ad6d-4e70-b121-563256c77d31 None admin] PATH_INFO: `/v3/users/350c85611b984015a23d8c9c7772914f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.903152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec44038-ad6d-4e70-b121-563256c77d31 None admin] RBAC: Authorizing `identity:get_user(user_id=350c85611b984015a23d8c9c7772914f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:51.904724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec44038-ad6d-4e70-b121-563256c77d31 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:51.906680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 87/177] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:51 2022] GET /identity/v3/users/350c85611b984015a23d8c9c7772914f => generated 275 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:51.907930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fec55-abf5-4d5e-9318-2d8d93a33c2e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.908215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fec55-abf5-4d5e-9318-2d8d93a33c2e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.908492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fec55-abf5-4d5e-9318-2d8d93a33c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.911198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c556f0-d492-4eee-86eb-563a8ba48362 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:51.939171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d94fec55-abf5-4d5e-9318-2d8d93a33c2e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:51.940384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d94fec55-abf5-4d5e-9318-2d8d93a33c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:51.950411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:51.951935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:51.952423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:51.952897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:51.953981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] RBAC: Adding query filter params (role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=350c85611b984015a23d8c9c7772914f) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:51.954509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:51.956230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:51.963231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 91/178] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:51 2022] GET /identity/v3/role_assignments?user.id=350c85611b984015a23d8c9c7772914f&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=36fb427222cd413e808b7d4028e53faf => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:51.964330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.117958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad272a-f924-4cdd-88b6-dd196b25a628 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.137826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 88/179] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:52.145198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c4ac1-201e-47d1-adec-80fe4bc09293 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.145526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c4ac1-201e-47d1-adec-80fe4bc09293 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.145776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c4ac1-201e-47d1-adec-80fe4bc09293 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.150672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 89/180] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:52.159119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.159382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.159684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.208621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:52.209862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:52.214630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9af97d-60c1-4300-b3e7-c53af914a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.215318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9af97d-60c1-4300-b3e7-c53af914a010 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.215602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9af97d-60c1-4300-b3e7-c53af914a010 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.215879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9af97d-60c1-4300-b3e7-c53af914a010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.244232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.313800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 90/181] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:52.314957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.319248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9af97d-60c1-4300-b3e7-c53af914a010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:52.320900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9af97d-60c1-4300-b3e7-c53af914a010 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:52.327862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.438539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6fc5b-89de-44f6-9e9d-ef74f0dcb8a5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.443483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 92/182] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:52.443483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966bcf-f2cc-4b80-8638-7cec755e86ee None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.443483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966bcf-f2cc-4b80-8638-7cec755e86ee None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.443483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966bcf-f2cc-4b80-8638-7cec755e86ee None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.443483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 93/183] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:52.447459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966bcf-f2cc-4b80-8638-7cec755e86ee None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.483862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.484662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.484948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.485225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.495515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:52.501298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:52.504210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:52.505225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:52.506007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 91/184] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:52.506892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.559346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966bcf-f2cc-4b80-8638-7cec755e86ee None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.587416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.594247 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824ea93-d1a0-4909-98ee-13411ec09931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.595071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824ea93-d1a0-4909-98ee-13411ec09931 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.595353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824ea93-d1a0-4909-98ee-13411ec09931 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.595626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824ea93-d1a0-4909-98ee-13411ec09931 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.605921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824ea93-d1a0-4909-98ee-13411ec09931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-8824ea93-d1a0-4909-98ee-13411ec09931 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:52.614180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:52.615635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:52.616405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 94/185] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:52.618755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.619151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.619446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.630769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e043d-54b3-4550-943e-bd94be796747 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.636087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:52.637328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:52.664993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc91489-25b3-41c4-8811-9b66687d43a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.665852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91489-25b3-41c4-8811-9b66687d43a8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.666140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91489-25b3-41c4-8811-9b66687d43a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.666421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc91489-25b3-41c4-8811-9b66687d43a8 None admin] PATH_INFO: `/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.667626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc91489-25b3-41c4-8811-9b66687d43a8 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe76eed4fd44bbca5de26adc977b9b2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:52.669084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc91489-25b3-41c4-8811-9b66687d43a8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:52.671437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 92/186] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/projects/bbe76eed4fd44bbca5de26adc977b9b2 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:52.672609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa299be-e4a4-49e8-a500-3255f024bca7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.673049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa299be-e4a4-49e8-a500-3255f024bca7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.673498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa299be-e4a4-49e8-a500-3255f024bca7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.674897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 93/187] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:52.679709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa299be-e4a4-49e8-a500-3255f024bca7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.705321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 95/188] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:52.705321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.749285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.785264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa299be-e4a4-49e8-a500-3255f024bca7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.806640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08e1a5-0d07-4238-9a85-dae570efcbe1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.814716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.820751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.821493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.821853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.822205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.823058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:52.823452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:52.824902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:52.834093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 94/189] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:52.835062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.841122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.841822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.842113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.842591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.843383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:52.843700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:52.845121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2169ef-b5b5-47cf-840b-68b612109911 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:52.854567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 96/190] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:52.855631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.855930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.856215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.876215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:52.877410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:52.949574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8fb82-992f-4ec6-a3eb-6fb2b175f1e2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:52.951050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 97/191] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:52.951883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:52.985924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:52.993786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:52.994440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:52.994725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:52.995031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:52.995865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=bbe76eed4fd44bbca5de26adc977b9b2, user.id=350c85611b984015a23d8c9c7772914f) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:52.996182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:52.997668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.015015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 95/192] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/role_assignments?user.id=350c85611b984015a23d8c9c7772914f&scope.project.id=bbe76eed4fd44bbca5de26adc977b9b2&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.015015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.067164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716470b-b372-4c74-9a7c-9011dabcfd4e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.079796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.109004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.109743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.110030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.110308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.122529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:53.129228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293188-1dc5-49e1-96cd-6701f77c1cb3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:53.132006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:53.132921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 98/193] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:49:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:53.139831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.170529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.171475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.171800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.172144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] PATH_INFO: `/v3/users/c01bcd18572a437d8fec306452e75ad9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.188569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] RBAC: Authorizing `identity:get_user(user_id=c01bcd18572a437d8fec306452e75ad9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.190352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.192252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 96/194] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/users/c01bcd18572a437d8fec306452e75ad9 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.192252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.254859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.264218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c0e7-c660-48d6-92be-7c7580b28248 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.304172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.305046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.305342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.305623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] PATH_INFO: `/v3/users/f91fa44198524e4ea0124a0f05b3d4b1` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.306829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=f91fa44198524e4ea0124a0f05b3d4b1)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.308349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.309994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 99/195] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/users/f91fa44198524e4ea0124a0f05b3d4b1 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.311160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.314092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9297c-39ea-44d7-9508-99f752ae7a53 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.329075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.349859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.350651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.350973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.351259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.352427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.353887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.360194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 97/196] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.360194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.412195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589f5ec-f74e-42e1-ad04-95b2750ef9c0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.446526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.447212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.447502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.447784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.448536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:53.448850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.450579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.454498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4b7ee-5852-4f8b-99e4-f0b6f30c2d99 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.458779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 100/197] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.459753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.487136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.489542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.490267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.490552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.490831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.541888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.560440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.561935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.564206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984bd70-3228-4b61-bfd6-2bf6b6600789 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.597664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.598559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.598860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.599149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.660262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 98/198] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:53.661014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.667142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.668681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.692057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.741700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.760549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1436-f93a-4bf0-b5a1-2caf83c00bdc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.769336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 101/199] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:53.770172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.802177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.802859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.803168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.803553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.804265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.805694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.826433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 99/200] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:49:53 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:53.827587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.878293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49217d-bf95-400b-86e5-f03cc2663ef7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.905226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:53.918046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.918869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.919165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.919478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] PATH_INFO: `/v3/users/c01bcd18572a437d8fec306452e75ad9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.920762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] RBAC: Authorizing `identity:get_user(user_id=c01bcd18572a437d8fec306452e75ad9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.922267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.928437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 102/201] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/users/c01bcd18572a437d8fec306452e75ad9 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.928437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:53.953502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868dedc-f4de-483d-aba9-d208d23c72b4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:53.991004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:53.991883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:53.992224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:53.992442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:53.993711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:53.995223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:53.996958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 100/202] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:53.999972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:54.014110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:54.049423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595fb66-1604-4eb8-b5aa-042437f89983 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:54.093323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:54.094250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:54.095060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:54.095353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:54.096576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:54.098162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:54.100176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 103/203] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:54.105773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:54.126908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc482fa-58d9-4aae-8504-233443658d15 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:54.176643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:54.177247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:54.177536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:54.177978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:54.183170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:54.232691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8681fa-4592-4e27-8d62-29ec98249a0a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:54.260553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:54.262579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:54.273235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:54.279461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:54.280419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:54.280714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:54.280999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:54.282016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=c01bcd18572a437d8fec306452e75ad9) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:54.282338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:54.284482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:54.291782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 104/204] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:54 2022] GET /identity/v3/role_assignments?user.id=c01bcd18572a437d8fec306452e75ad9&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:54.292672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:54.407107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 101/205] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:54.412727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:54.421236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ad8c8-7abc-4061-9cdf-7396e54e3812 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:54.449026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:54.464573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:54.465713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:54.466751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:54.467311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:54.468369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] RBAC: Adding query filter params (role.id=881f45605c544dddb6830e6749387c9d, scope.project.id=6fced2c17e3741e992d8466df4bd6539, user.id=f91fa44198524e4ea0124a0f05b3d4b1) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:54.468889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:54.470865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:54.478248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 105/206] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:54 2022] GET /identity/v3/role_assignments?user.id=f91fa44198524e4ea0124a0f05b3d4b1&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=881f45605c544dddb6830e6749387c9d => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:54.547973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a3763-4a90-49d7-9a4e-a8e5dc1a629d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:54.556670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:54.601297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d44e1a-71b4-4b87-88a2-5ec5231dfee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:54.602641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44e1a-71b4-4b87-88a2-5ec5231dfee0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:54.603239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44e1a-71b4-4b87-88a2-5ec5231dfee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:54.603722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44e1a-71b4-4b87-88a2-5ec5231dfee0 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:54.605392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44e1a-71b4-4b87-88a2-5ec5231dfee0 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:54.607496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44e1a-71b4-4b87-88a2-5ec5231dfee0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:54.609990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 102/207] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:54 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:54.616455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:54.734088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2708074-1867-4fff-869f-bd2a23f318e3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:54.771919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:54.772992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:54.773492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:54.774042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:54.775101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] RBAC: Adding query filter params (role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=c01bcd18572a437d8fec306452e75ad9) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:54.775641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:54.777371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613ca3c-4a76-4f47-8156-0cc9c07f373f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:54.784191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 106/208] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:54 2022] GET /identity/v3/role_assignments?user.id=c01bcd18572a437d8fec306452e75ad9&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=36fb427222cd413e808b7d4028e53faf => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:56.453311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5c318-2195-419e-8267-ef76711d178e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.453823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5c318-2195-419e-8267-ef76711d178e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.453898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5c318-2195-419e-8267-ef76711d178e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.455437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 103/209] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:56.461459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.461459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.461459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.479377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:56.480867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:56.499142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d60918-e6db-4c81-a523-0cf7602906be None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.499707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d60918-e6db-4c81-a523-0cf7602906be None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.500210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d60918-e6db-4c81-a523-0cf7602906be None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.501699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 104/210] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:56.507688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.507975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.508251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.527813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:56.529919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:56.536812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:56.558693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 107/211] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:56.582294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:56.604656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 105/212] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:56.632078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:56.698745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116fdca-b937-4898-9bf2-28e64e9715c7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:56.736697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:56.737725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.738152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.738568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.750702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:56.758657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:56.759543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:49:56.760333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:49:56.761411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0dce7-9c16-40fa-9f8f-66401cf6dfd3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:56.763701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:56.763942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 108/213] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:49:56.774873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:56.810573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:56.811824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.812233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.812903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.814378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:56.817018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:56.822928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 106/214] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:56.824306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:56.827191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:56.893360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56187d-f867-47d6-8d57-9f8bac78eaf8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:56.929817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:56.930872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.931360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.931835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.932777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:56.933301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:56.934173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1604-23ad-4b01-985c-eb4723233374 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:56.935831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:56.944977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 109/215] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:49:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:56.950551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:56.977126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826ca20-8a4b-4616-a609-8004d2a335f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:56.978144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826ca20-8a4b-4616-a609-8004d2a335f3 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:56.978434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826ca20-8a4b-4616-a609-8004d2a335f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:56.978713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826ca20-8a4b-4616-a609-8004d2a335f3 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:56.979511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826ca20-8a4b-4616-a609-8004d2a335f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:56.981006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826ca20-8a4b-4616-a609-8004d2a335f3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:57.004620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 107/216] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:49:56 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:49:57.006416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e96f8-514f-4a82-9537-9be7f58a75e1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.006810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e96f8-514f-4a82-9537-9be7f58a75e1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.007126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e96f8-514f-4a82-9537-9be7f58a75e1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.008732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 108/217] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:57.014509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.015117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.034323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:57.035599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:57.064484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.107957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe91189-9c78-427d-a018-1748e1334844 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.110671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.121676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 109/218] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:57.145796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.151619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.152526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.152809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.153084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] PATH_INFO: `/v3/users/e7d0e6aaa23c4d9f88fc7cb461f98324` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.173169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e7d0e6aaa23c4d9f88fc7cb461f98324)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:57.175457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:57.177818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 110/219] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:56 2022] GET /identity/v3/users/e7d0e6aaa23c4d9f88fc7cb461f98324 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:57.183448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.263914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f597d-2865-4c79-a264-64bf6b82f5ad None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.298307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e994aa-5cc9-43d5-9e7f-999db34b54f5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.313820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd02c1-7db5-43a8-970d-78afd9a84312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.314806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd02c1-7db5-43a8-970d-78afd9a84312 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.315068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd02c1-7db5-43a8-970d-78afd9a84312 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.315339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd02c1-7db5-43a8-970d-78afd9a84312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.329653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd02c1-7db5-43a8-970d-78afd9a84312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-10dd02c1-7db5-43a8-970d-78afd9a84312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:57.338883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:57.339755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:57.340585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 110/220] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:57.342059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb71a3-cfda-4ee7-9e33-cebde38f75c0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.342059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb71a3-cfda-4ee7-9e33-cebde38f75c0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.342059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb71a3-cfda-4ee7-9e33-cebde38f75c0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.342278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b09481-787c-4a24-9931-589c89af5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.343170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b09481-787c-4a24-9931-589c89af5200 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.343369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b09481-787c-4a24-9931-589c89af5200 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.343653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b09481-787c-4a24-9931-589c89af5200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.346122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 111/221] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:57.348735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb71a3-cfda-4ee7-9e33-cebde38f75c0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.381367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.387046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.434689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b09481-787c-4a24-9931-589c89af5200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:57.436986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b09481-787c-4a24-9931-589c89af5200 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:57.494095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb71a3-cfda-4ee7-9e33-cebde38f75c0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.540507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.541517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.542145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.542446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.543484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:57.544451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:57.546393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:57.551320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 111/222] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:49:57.552718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.553078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.553370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.561285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 112/223] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:57.562223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.571815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:57.573242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:57.642314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.657498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 112/224] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:49:57.658020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.659105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.694568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb0b9c-cb65-4a34-b9ba-27616b80b501 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.737362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.738533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.739028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.739734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.741834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:57.744042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:57.747840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 113/225] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:57.748926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.777273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838072f1-c506-4b69-8561-402a6f24c1be None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.820560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.821716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.821997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.822518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] PATH_INFO: `/v3/users/c01bcd18572a437d8fec306452e75ad9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.824266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=c01bcd18572a437d8fec306452e75ad9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:57.826278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:57.828797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 113/226] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/users/c01bcd18572a437d8fec306452e75ad9 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:57.829397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.867161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6d456-e184-499c-a081-6b00ad3f9f4e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:57.906337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:57.907501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:57.907891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:57.908269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:57.909865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.920562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:49:57.929237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:49:57.930124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:49:57.930956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:49:57.936029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 114/227] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:49:57.936962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:57.949574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:57.963990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bf0a5-51cc-45a9-81c5-10567ddfa4b0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.017160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.017605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.017892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.018174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.019042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=e7d0e6aaa23c4d9f88fc7cb461f98324) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:58.019363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.020829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:58.026595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 114/228] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/role_assignments?user.id=e7d0e6aaa23c4d9f88fc7cb461f98324&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:58.030998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:58.098149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a057-e43d-43aa-a118-0d3f0d109995 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.172256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.173451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.175190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.175658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.180207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.181943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:58.184007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 115/229] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:57 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:58.189935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:58.238384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f50c7-a0aa-4b69-9b75-03266462423e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.282209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:58.301446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.302433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.302951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.303429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.304411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:49:58.304936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.306532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:58.320876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:58.325154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 115/230] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:49:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:58.331259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:58.349196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4e35-fc43-4205-8cf0-d05d0ddfecb5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.393238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.394156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.394441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.394713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/c01bcd18572a437d8fec306452e75ad9/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.404003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] RBAC: Authorizing `identity:create_grant(project_id=ec71c3cb2fa34334b0a06778fa4f9800, user_id=c01bcd18572a437d8fec306452e75ad9, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.405520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:58.430870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 116/231] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:58 2022] PUT /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/c01bcd18572a437d8fec306452e75ad9/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:58.523395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be7acc-2506-4f3d-9b04-fa2d5ad38f2e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.572971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.573870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.574161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.574444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] PATH_INFO: `/v3/users/c01bcd18572a437d8fec306452e75ad9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.575795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] RBAC: Authorizing `identity:get_user(user_id=c01bcd18572a437d8fec306452e75ad9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.577303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:58.579317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 116/232] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:49:58 2022] GET /identity/v3/users/c01bcd18572a437d8fec306452e75ad9 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:49:58.584916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:58.738881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff451-acf2-475d-a51e-c5751d690f5f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.775259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a5547-bddb-49c9-a295-2c7876b7e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.776488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5547-bddb-49c9-a295-2c7876b7e422 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.776991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5547-bddb-49c9-a295-2c7876b7e422 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.777465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a5547-bddb-49c9-a295-2c7876b7e422 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.779214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a5547-bddb-49c9-a295-2c7876b7e422 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.780913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a5547-bddb-49c9-a295-2c7876b7e422 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:58.782947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 117/233] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:49:58 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:49:58.789559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:58.870947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:58.937783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882583-dadb-4fff-8e95-6f33d68895dd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:49:58.986145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae937-20e2-4e5d-ace8-6f7013b077d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:49:58.987484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae937-20e2-4e5d-ace8-6f7013b077d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:58.987972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae937-20e2-4e5d-ace8-6f7013b077d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:58.988448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae937-20e2-4e5d-ace8-6f7013b077d1 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/c01bcd18572a437d8fec306452e75ad9/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:58.995385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ae937-20e2-4e5d-ace8-6f7013b077d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec71c3cb2fa34334b0a06778fa4f9800, user_id=c01bcd18572a437d8fec306452e75ad9, role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:49:58.998185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ae937-20e2-4e5d-ace8-6f7013b077d1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:49:59.021728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 117/234] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:49:58 2022] PUT /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/c01bcd18572a437d8fec306452e75ad9/roles/36fb427222cd413e808b7d4028e53faf => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:49:59.712496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236730-6d22-46c7-8aee-56e9c166aea2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:59.713008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236730-6d22-46c7-8aee-56e9c166aea2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:59.713110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236730-6d22-46c7-8aee-56e9c166aea2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:59.714857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 118/235] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:49:59.721258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033289d-4c6f-4ddc-b530-18e165e119d8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:59.721677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033289d-4c6f-4ddc-b530-18e165e119d8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:59.721851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033289d-4c6f-4ddc-b530-18e165e119d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:59.740504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c033289d-4c6f-4ddc-b530-18e165e119d8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:59.741837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c033289d-4c6f-4ddc-b530-18e165e119d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:49:59.819060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 118/236] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:49:59.839576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236730-6d22-46c7-8aee-56e9c166aea2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:49:59.844292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:49:59.868903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00bbd-dc5f-4d89-a734-951ac539b108 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:59.869213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00bbd-dc5f-4d89-a734-951ac539b108 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:59.869475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00bbd-dc5f-4d89-a734-951ac539b108 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:59.870885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 119/237] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:49:59.878951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:49:59.879248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:49:59.879535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:49:59.913048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:49:59.915194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:00.000278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236730-6d22-46c7-8aee-56e9c166aea2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:00.007415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:00.043964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:00.045014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.045301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.045581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.047782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:00.049251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:00.050899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 119/238] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:00.060812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:00.080489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 120/239] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:00.107273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:00.191826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:00.255189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8f56a-5db7-4912-8ecc-c03058b83a71 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:00.263961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda26b8-ef54-4f77-86b5-3ef2d5183edd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:00.309539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:00.310223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.310513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.310799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.311544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:00.313009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:00.315381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:00.316188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.316470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.316748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.336935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:00.346527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:00.347705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:00.348482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 121/240] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:00.358509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 120/241] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:00 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:50:00.359062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:00.459254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:00.506811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd3c0-99dc-494e-ae1a-c8a22b49b6d8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:00.546637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:00.547364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.547654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.547937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.548722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:00.549038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:00.550526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:00.564720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 122/242] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:00.570568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:00.699336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87fcbe-2a73-4d8a-8a2c-e403978695ae None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:00.727815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:00.735840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcae15-d523-426b-ba1c-24410a2d3861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:00.737083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcae15-d523-426b-ba1c-24410a2d3861 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.737556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcae15-d523-426b-ba1c-24410a2d3861 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.737842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcae15-d523-426b-ba1c-24410a2d3861 None admin] PATH_INFO: `/v3/users/ec834e28e5f040b5a7f7daf36c7db9fe` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.754084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcae15-d523-426b-ba1c-24410a2d3861 None admin] RBAC: Authorizing `identity:get_user(user_id=ec834e28e5f040b5a7f7daf36c7db9fe)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:00.755600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcae15-d523-426b-ba1c-24410a2d3861 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:00.757391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 121/243] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:00 2022] GET /identity/v3/users/ec834e28e5f040b5a7f7daf36c7db9fe => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:00.762883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:00.879764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:00.890262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082d2e3-0053-46dc-81b9-ebba05308a51 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:00.915488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747bf15-2eaf-4ae8-9cf8-7836d02b166d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.916108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747bf15-2eaf-4ae8-9cf8-7836d02b166d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.916608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747bf15-2eaf-4ae8-9cf8-7836d02b166d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.919066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 122/244] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:50:00.928474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.929091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.929590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.929699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:00.930298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:00.930592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:00.930904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:00.959312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:00.960925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:01.037040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:01.038543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:01.071254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 123/245] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:01.093019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.139668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:01.148037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 123/246] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:01.153797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.209391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb67b8-5059-43cf-9136-3eda10c18614 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:01.231785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:01.254359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:01.255547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.256032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.256521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.261244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df936c-c15e-436d-98c1-a8a2a91929b7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:01.267854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:01.275799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:01.276677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:01.278004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:01.280701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 124/247] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:01.286500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.298163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:01.299063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.299347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.299621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.300853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:01.302336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:01.304321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 124/248] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:01.310938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.387827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:01.411633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15382e77-c386-44b3-bd87-8aa385251cd4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:01.428995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccd058-9984-4acc-b25a-ebcca20d198c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:01.490447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:01.491230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.491531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.491812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.492574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:01.492885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:01.494298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea270be8-fe7d-4fe1-a109-91616606e337 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:01.500978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:01.501649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.501939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.502215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.503084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] RBAC: Adding query filter params (role.id=881f45605c544dddb6830e6749387c9d, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=ec834e28e5f040b5a7f7daf36c7db9fe) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:01.503403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:01.504885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701ad63-9d07-4397-96e5-4523350b5cc1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:01.514890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 125/249] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:01.517083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697003-fbec-4909-a1bd-4803cb232ca1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.517381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697003-fbec-4909-a1bd-4803cb232ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.517667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697003-fbec-4909-a1bd-4803cb232ca1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.519011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 126/250] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:01.520110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70697003-fbec-4909-a1bd-4803cb232ca1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.521172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 125/251] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/role_assignments?user.id=ec834e28e5f040b5a7f7daf36c7db9fe&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=881f45605c544dddb6830e6749387c9d => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:01.527020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.527906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.529092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.575592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:01.577246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:01.621349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:01.752478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 126/252] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:01.763127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70697003-fbec-4909-a1bd-4803cb232ca1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:01.794324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.811972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:01.812858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.813150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.813434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] PATH_INFO: `/v3/users/e7d0e6aaa23c4d9f88fc7cb461f98324` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.819669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] RBAC: Authorizing `identity:get_user(user_id=e7d0e6aaa23c4d9f88fc7cb461f98324)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:01.821311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:01.824174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 127/253] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/users/e7d0e6aaa23c4d9f88fc7cb461f98324 => generated 297 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:01.828652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:01.877444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:01.910646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe010a52-8683-4dee-8856-7e3c66193b8b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:01.949578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:01.950751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:01.951266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:01.951791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:01.976189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:01.993054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:01.994007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:01.994779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:02.000281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 127/254] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:02.007480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:02.017547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af6a1-965b-480c-bf62-70132827c611 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:02.030910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:02.082867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3815e-a4a9-4742-ad8d-d3879db98887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:02.082867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3815e-a4a9-4742-ad8d-d3879db98887 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.082867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3815e-a4a9-4742-ad8d-d3879db98887 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.082867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3815e-a4a9-4742-ad8d-d3879db98887 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.083312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3815e-a4a9-4742-ad8d-d3879db98887 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:02.083312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3815e-a4a9-4742-ad8d-d3879db98887 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:02.083312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 128/255] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:01 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:02.083528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc05b-e704-4c1b-b488-92214737fc8b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.083753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc05b-e704-4c1b-b488-92214737fc8b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.084034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc05b-e704-4c1b-b488-92214737fc8b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.085294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 129/256] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:02.088066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc05b-e704-4c1b-b488-92214737fc8b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:02.203597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc05b-e704-4c1b-b488-92214737fc8b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:02.240352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92b643-06e4-4db6-9006-a239190dfa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:02.241340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b643-06e4-4db6-9006-a239190dfa86 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.241703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b643-06e4-4db6-9006-a239190dfa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.242049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b643-06e4-4db6-9006-a239190dfa86 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/e7d0e6aaa23c4d9f88fc7cb461f98324/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.246860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b643-06e4-4db6-9006-a239190dfa86 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec71c3cb2fa34334b0a06778fa4f9800, user_id=e7d0e6aaa23c4d9f88fc7cb461f98324, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:02.246860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b643-06e4-4db6-9006-a239190dfa86 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:02.262826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 130/257] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:02 2022] PUT /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/e7d0e6aaa23c4d9f88fc7cb461f98324/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 11:50:02.264704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.265064 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.265424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.285084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52964f26-fcf2-493a-9f49-eab8e6fdb3c9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:02.286176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:02.287677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:02.318211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:02.323499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:02.369271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:02.369886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.370176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.370458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.371374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:02.371762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:02.373274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:02.398228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 128/258] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:02.405129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:02.431231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 131/259] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:02.482160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:02.584137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276133e7-174d-4fd0-8b3f-cf1e9a9a771c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:02.622669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:02.623523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.623809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.624083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] PATH_INFO: `/v3/users/c01bcd18572a437d8fec306452e75ad9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.625697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] RBAC: Authorizing `identity:get_user(user_id=c01bcd18572a437d8fec306452e75ad9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:02.627499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:02.629338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 129/260] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:02 2022] GET /identity/v3/users/c01bcd18572a437d8fec306452e75ad9 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:02.634945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:02.690407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:02.756252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff107e-e093-4fc7-bda3-c8b13df56f59 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:02.773614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0efad-9f55-4102-b661-12e676d0bb31 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:02.799160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:02.800531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.801043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.801546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.868151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:02.868982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:02.869269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:02.869549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:02.896111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:02.912788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:02.912992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:02.913848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:02.915305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:02.916005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:02.916073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 132/261] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:02.919853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:02.979823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:02.991197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:03.088216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 130/262] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:03.093422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:03.180848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a6a61-118e-4253-9a11-36c48671be8d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:03.243738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811f4a8-8632-462a-a724-b2f107e1744d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:03.296168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:03.296885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:03.297288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:03.297599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:03.298369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:03.298681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:03.300115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:03.300238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:03.300983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:03.301267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:03.301542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:03.307576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:03.309154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:03.311130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 131/263] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:03 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:03.316678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:03.318500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 133/264] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:03.324885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:03.347424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:03.485723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cc33-857d-4859-8a45-85010b7c731b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:03.535245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:03.551549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:03.552221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:03.552508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:03.552611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4d14b-500a-4b1f-af7d-214345ab270a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:03.552789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:03.553633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=c01bcd18572a437d8fec306452e75ad9) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:03.553949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:03.555448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:03.567054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 132/265] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:03 2022] GET /identity/v3/role_assignments?user.id=c01bcd18572a437d8fec306452e75ad9&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:03.628129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:03.629027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:03.629314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:03.629595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] PATH_INFO: `/v3/users/c01bcd18572a437d8fec306452e75ad9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:03.631546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] RBAC: Authorizing `identity:get_user(user_id=c01bcd18572a437d8fec306452e75ad9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:03.633024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:03.634919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 134/266] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:03 2022] GET /identity/v3/users/c01bcd18572a437d8fec306452e75ad9 => generated 283 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:03.640806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:03.778633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:03.818480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa232e-3afe-49f3-906f-746ece4f5680 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:03.887707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdd025-48cd-408f-b7cc-15e238a720ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:03.888798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdd025-48cd-408f-b7cc-15e238a720ea None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:03.889285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdd025-48cd-408f-b7cc-15e238a720ea None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:03.889793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdd025-48cd-408f-b7cc-15e238a720ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:03.969755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfdd025-48cd-408f-b7cc-15e238a720ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:03.971672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfdd025-48cd-408f-b7cc-15e238a720ea None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:04.091721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:04.122522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 133/267] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:04.128548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:04.207830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc41db-74e3-43d3-8c4a-31f1fb5df4d9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.208128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc41db-74e3-43d3-8c4a-31f1fb5df4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.208400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc41db-74e3-43d3-8c4a-31f1fb5df4d9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.211336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 134/268] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:04.215427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.215720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.216005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.232793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:04.234005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:04.261103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd430e41-d044-4ca5-88ea-6091038466b5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:04.319327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 135/269] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:04.343526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:04.388168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:04.389062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.389349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.389631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.390856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:04.392356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:04.394344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 135/270] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:04 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:04.403844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:04.458086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:04.461670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f601ec-50cd-47af-8494-f7d5bae089ea None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:04.499237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:04.514518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01532af-ef84-438e-b88c-f0515080f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:04.515355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01532af-ef84-438e-b88c-f0515080f463 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.515639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01532af-ef84-438e-b88c-f0515080f463 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.515916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01532af-ef84-438e-b88c-f0515080f463 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.517103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01532af-ef84-438e-b88c-f0515080f463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:04.518603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01532af-ef84-438e-b88c-f0515080f463 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:04.520251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 136/271] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:04.523612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539f15-977f-47cd-ba11-ce2ca531e04e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.523612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539f15-977f-47cd-ba11-ce2ca531e04e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.523612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539f15-977f-47cd-ba11-ce2ca531e04e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.523612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 137/272] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:04.529802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.530057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.530346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.548824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:04.550021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:04.558294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adcc57-f0d0-4bb4-83d6-f721729021f3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:04.610931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:04.611945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.612432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.613568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.614973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] RBAC: Adding query filter params (role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=c01bcd18572a437d8fec306452e75ad9) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:04.615544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:04.618258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:04.627503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 136/273] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:04 2022] GET /identity/v3/role_assignments?user.id=c01bcd18572a437d8fec306452e75ad9&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=36fb427222cd413e808b7d4028e53faf => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:04.628673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:04.635089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 138/274] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:04.652783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:04.737670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:04.757993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638099c-75fe-4cba-9251-542d49531c3c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:04.764075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52552423-e8c1-404e-9469-b6682b31e5d3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:04.772105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:04.801618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:04.802417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.802919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.803415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.820707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bebeb9-a625-4fb0-9b1a-9d3de0f31e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:04.821796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bebeb9-a625-4fb0-9b1a-9d3de0f31e5c None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:04.822507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bebeb9-a625-4fb0-9b1a-9d3de0f31e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:04.824853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bebeb9-a625-4fb0-9b1a-9d3de0f31e5c None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:04.825953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bebeb9-a625-4fb0-9b1a-9d3de0f31e5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:04.828270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bebeb9-a625-4fb0-9b1a-9d3de0f31e5c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:04.841939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:04.851009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:04.851982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:04.852766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:04.856764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 139/275] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:04.864932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:04.879029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 137/276] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:04 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:04.996426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30889fd-6bc0-4a38-8b4c-3abdd3158637 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.040334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe34653-dc80-426e-bfdb-64a52611f7d3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.040753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe34653-dc80-426e-bfdb-64a52611f7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.041118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe34653-dc80-426e-bfdb-64a52611f7d3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.047040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 138/277] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:05.048768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.049275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.049759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.050799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.052024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.052024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.052157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.052830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:05.053146 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:05.055721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:05.068387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 140/278] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:05.072153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.085589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:05.086820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:05.120144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:05.175932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 139/279] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:05.199022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.212415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc443a-48c4-4dba-ab52-0739bff207cc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.251005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4801b8-a7e2-4d5e-8970-46c61008596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.252157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4801b8-a7e2-4d5e-8970-46c61008596b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.252599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4801b8-a7e2-4d5e-8970-46c61008596b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.252883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4801b8-a7e2-4d5e-8970-46c61008596b None admin] PATH_INFO: `/v3/users/f730cc219f4247f98d0b291fa9774c4e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.267577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4801b8-a7e2-4d5e-8970-46c61008596b None admin] RBAC: Authorizing `identity:get_user(user_id=f730cc219f4247f98d0b291fa9774c4e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:05.269069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4801b8-a7e2-4d5e-8970-46c61008596b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:05.270800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 141/280] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/users/f730cc219f4247f98d0b291fa9774c4e => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:05.273112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a7a53-2b63-41e5-bd71-024a50a41de2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.273112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a7a53-2b63-41e5-bd71-024a50a41de2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.273112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a7a53-2b63-41e5-bd71-024a50a41de2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.274000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 142/281] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:05.280170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a7a53-2b63-41e5-bd71-024a50a41de2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.293061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:05.336765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b187-ca19-45f1-b5be-ef62cc955960 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.385362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a11b63-adfc-4142-ae85-f12960bbe8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.387510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a11b63-adfc-4142-ae85-f12960bbe8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.388216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a11b63-adfc-4142-ae85-f12960bbe8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.388977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a11b63-adfc-4142-ae85-f12960bbe8e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.392330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:05.402795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a11b63-adfc-4142-ae85-f12960bbe8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:05.406348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a7a53-2b63-41e5-bd71-024a50a41de2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-61a11b63-adfc-4142-ae85-f12960bbe8e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:05.412364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:05.413251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:05.414036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:05.416748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 140/282] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:05.422576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.423352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.423870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.445582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:05.446704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.447437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:05.447552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.447956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.448326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.450082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, user.id=f730cc219f4247f98d0b291fa9774c4e, scope.system=all) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:05.450516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:05.452369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:05.458207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 143/283] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/role_assignments?user.id=f730cc219f4247f98d0b291fa9774c4e&scope.system=all&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:05.459098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.570990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 141/284] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:05.611763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.625418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bcb5f-b461-4534-9e2c-95a78585866a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.669548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.670181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.670525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.670814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.671596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:05.671912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:05.673324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:05.682413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 144/285] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:05.688293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.695179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:05.721353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:05.778989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7b746-0957-4b36-b89d-42611bfcdd59 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.850182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.851012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.851303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.851586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.854091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318b23-10a4-4442-b313-c67d4e85bd90 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:05.874351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:05.885438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:05.886469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:05.887353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:05.888438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 142/286] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:05.900698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:05.922587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:05.923815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:05.925697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:05.926193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] PATH_INFO: `/v3/users/ec834e28e5f040b5a7f7daf36c7db9fe` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:05.928339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] RBAC: Authorizing `identity:get_user(user_id=ec834e28e5f040b5a7f7daf36c7db9fe)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:05.930186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:05.936201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 145/287] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/users/ec834e28e5f040b5a7f7daf36c7db9fe => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:05.939517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:06.090412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b59ba-24a6-4b4f-bc2a-06ac2beb428b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:06.094311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:06.129890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d757-c077-4026-9c89-31dbd3524a32 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:06.131178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:06.149920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:06.150598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:06.150911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:06.151213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:06.152293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:06.152622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:06.154020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:06.179463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 143/288] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:06.181197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:06.201917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:06.202790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:06.204385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:06.205379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:06.207703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:06.209821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:06.213732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 146/289] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:05 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:06.218005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:06.365694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1331df3-44fa-4c56-9375-bf0501a17e42 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:06.382014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fe0bb-0b84-4a44-a831-26335cdc879a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:06.427459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:06.428353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:06.428650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:06.428936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] PATH_INFO: `/v3/users/e7d0e6aaa23c4d9f88fc7cb461f98324` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:06.430333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] RBAC: Authorizing `identity:get_user(user_id=e7d0e6aaa23c4d9f88fc7cb461f98324)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:06.431800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:06.437869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 144/290] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:06 2022] GET /identity/v3/users/e7d0e6aaa23c4d9f88fc7cb461f98324 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:06.442796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:06.443307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:06.444432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:06.444892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:06.445311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/ec834e28e5f040b5a7f7daf36c7db9fe/roles/881f45605c544dddb6830e6749387c9d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:06.447566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec71c3cb2fa34334b0a06778fa4f9800, user_id=ec834e28e5f040b5a7f7daf36c7db9fe, role_id=881f45605c544dddb6830e6749387c9d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:06.450263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:06.477869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 147/291] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:06 2022] PUT /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/users/ec834e28e5f040b5a7f7daf36c7db9fe/roles/881f45605c544dddb6830e6749387c9d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:06.543062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:06.619002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057850c4-f44f-443c-b946-36dcaaea5294 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:06.667783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:06.668490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:06.668786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:06.669068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:06.758069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:06.759519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:06.838200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:06.900364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 145/292] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:06.906387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:06.933413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:07.036442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cbcb-ebf0-439b-abea-1bc9f57bf926 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:07.071328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e6dd1-c238-45d4-897a-7770ab406d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:07.072201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e6dd1-c238-45d4-897a-7770ab406d7f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:07.072489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e6dd1-c238-45d4-897a-7770ab406d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:07.072769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e6dd1-c238-45d4-897a-7770ab406d7f None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:07.074495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e6dd1-c238-45d4-897a-7770ab406d7f None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:07.075994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e6dd1-c238-45d4-897a-7770ab406d7f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:07.082514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 148/293] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:06 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:07.083946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:07.242272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa79d-f473-42de-9d43-b3665848081a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:07.278253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:07.278857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:07.279183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:07.279473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:07.280316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=e7d0e6aaa23c4d9f88fc7cb461f98324) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:07.280635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:07.282083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19055e24-16be-4f75-9582-a824df158bb8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:07.288115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 146/294] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:07 2022] GET /identity/v3/role_assignments?user.id=e7d0e6aaa23c4d9f88fc7cb461f98324&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:07.717940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2f9a0-0cd7-4058-81be-8aef7061ff59 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:07.718308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2f9a0-0cd7-4058-81be-8aef7061ff59 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:07.718518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2f9a0-0cd7-4058-81be-8aef7061ff59 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:07.720300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 149/295] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:07.726110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba745b6-4a6c-4b26-88ad-5d25b0eabdb1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:07.726419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba745b6-4a6c-4b26-88ad-5d25b0eabdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:07.726713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba745b6-4a6c-4b26-88ad-5d25b0eabdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:07.745783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3ba745b6-4a6c-4b26-88ad-5d25b0eabdb1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:07.747061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3ba745b6-4a6c-4b26-88ad-5d25b0eabdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:07.762282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:07.856609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 147/296] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:07.884351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2f9a0-0cd7-4058-81be-8aef7061ff59 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:07.948727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08d551-0372-4a2d-9cdc-70d9b86dfe99 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:07.949073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08d551-0372-4a2d-9cdc-70d9b86dfe99 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:07.949358 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08d551-0372-4a2d-9cdc-70d9b86dfe99 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:07.950743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 148/297] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:50:07.966488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:07.967114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:07.967601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:07.982296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:07.993515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:07.996338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:08.014567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2f9a0-0cd7-4058-81be-8aef7061ff59 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.061733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.062883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.063357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.063784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.065135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:08.066867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:08.069036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 150/298] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:08.074812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.094872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 149/299] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:08.121554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.210314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e408b8-a1b8-4498-b54f-a5090a458901 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.212188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:08.238186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427826d-c008-46cc-9d49-5382a82a22dd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.247582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fc303-eb04-4e16-88c4-4dcb704d9979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.248590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fc303-eb04-4e16-88c4-4dcb704d9979 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.249018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fc303-eb04-4e16-88c4-4dcb704d9979 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.249460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fc303-eb04-4e16-88c4-4dcb704d9979 None admin] PATH_INFO: `/v3/groups` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.250333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fc303-eb04-4e16-88c4-4dcb704d9979 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:08.251966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fc303-eb04-4e16-88c4-4dcb704d9979 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:08.269136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 151/300] 10.222.0.41 () {60 vars in 1134 bytes} [Mon Jun 6 11:50:08 2022] POST /identity/v3/groups => generated 229 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:08.270736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9020-96a6-41d8-aab7-c4537784f1b5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.271077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9020-96a6-41d8-aab7-c4537784f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.271337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9020-96a6-41d8-aab7-c4537784f1b5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.272783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 152/301] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:08.280726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.281009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.281299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.283246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef951d-c700-4202-805e-336d6947835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.284795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef951d-c700-4202-805e-336d6947835a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.285301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef951d-c700-4202-805e-336d6947835a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.285765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef951d-c700-4202-805e-336d6947835a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.298453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:08.298897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef951d-c700-4202-805e-336d6947835a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:08.300303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:08.301613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-82ef951d-c700-4202-805e-336d6947835a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:08.307213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:08.309178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:08.310008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 150/302] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:08.317752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef951d-c700-4202-805e-336d6947835a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.389316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 153/303] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:08.420828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.454184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef951d-c700-4202-805e-336d6947835a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.495009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.495871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.496157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.496433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.497186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:08.497494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:08.500505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:08.504013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:08.514494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 151/304] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:08.520492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.560058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef03719-436e-47d2-af3f-2d8886facd90 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.599449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.600344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.600697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.601043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.603320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:08.613920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:08.622427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:08.623348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:08.624433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 154/305] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:08.629653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.654580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e190a2a-1f98-4e73-91df-91ecba4f6e0e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.696496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.697497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.697843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.698971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] PATH_INFO: `/v3/users/502fab24c7924de2bbb97cb051a616c0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.713642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] RBAC: Authorizing `identity:get_user(user_id=502fab24c7924de2bbb97cb051a616c0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:08.715847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:08.718258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 152/306] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/users/502fab24c7924de2bbb97cb051a616c0 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:08.723979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.758750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bd314-370d-4311-a376-6705a57f8773 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.784826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:08.808485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.809493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.809946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.810397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.811363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:08.811837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:08.813731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:08.831294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 155/307] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:08.837832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:08.850096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e2de-358d-4190-9117-627d81d4ec43 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:08.896949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:08.897991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:08.898118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:08.898482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:08.899461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] RBAC: Adding query filter params (role.id=881f45605c544dddb6830e6749387c9d, user.id=502fab24c7924de2bbb97cb051a616c0, scope.system=all) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:08.899982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:08.902477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:08.906439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:08.909554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 153/308] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/role_assignments?user.id=502fab24c7924de2bbb97cb051a616c0&scope.system=all&role.id=881f45605c544dddb6830e6749387c9d => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:08.963429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b3d9f-f8b9-4158-bff2-e881e630a62d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:09.009382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e4641-0541-4288-858b-49849b7974fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:09.011224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4641-0541-4288-858b-49849b7974fe None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.011670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4641-0541-4288-858b-49849b7974fe None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.012102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4641-0541-4288-858b-49849b7974fe None admin] PATH_INFO: `/v3/users/f730cc219f4247f98d0b291fa9774c4e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.013711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e4641-0541-4288-858b-49849b7974fe None admin] RBAC: Authorizing `identity:get_user(user_id=f730cc219f4247f98d0b291fa9774c4e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:09.015430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e4641-0541-4288-858b-49849b7974fe None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:09.020681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 156/309] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:08 2022] GET /identity/v3/users/f730cc219f4247f98d0b291fa9774c4e => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:09.023863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:09.138302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acfc7d-9c0c-4f46-8197-8a49acef8c97 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:09.167998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:09.186471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8551156-679b-45ec-96c7-11ff0b827474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:09.188102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8551156-679b-45ec-96c7-11ff0b827474 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.188637 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8551156-679b-45ec-96c7-11ff0b827474 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.189163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8551156-679b-45ec-96c7-11ff0b827474 None admin] PATH_INFO: `/v3/system/users/f730cc219f4247f98d0b291fa9774c4e/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.190981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8551156-679b-45ec-96c7-11ff0b827474 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f730cc219f4247f98d0b291fa9774c4e, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:09.193019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8551156-679b-45ec-96c7-11ff0b827474 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:09.204196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 154/310] 10.222.0.41 () {58 vars in 1334 bytes} [Mon Jun 6 11:50:09 2022] PUT /identity/v3/system/users/f730cc219f4247f98d0b291fa9774c4e/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:09.350533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f5831-3f3e-4c2b-a2ab-281fe8e7a27a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.350934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f5831-3f3e-4c2b-a2ab-281fe8e7a27a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.351157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f5831-3f3e-4c2b-a2ab-281fe8e7a27a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.352556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 157/311] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:09.359201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.359732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.359988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.382193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:09.383823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:09.465176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 155/312] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:09.488685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f5831-3f3e-4c2b-a2ab-281fe8e7a27a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:09.605223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f5831-3f3e-4c2b-a2ab-281fe8e7a27a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:09.634450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:09.646572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:09.647397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.647689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.647971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.659206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:09.667154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:09.668053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:09.668843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 158/313] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:09.674202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:09.781212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99b341-af28-47af-8f16-34f8f51a64ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:09.796523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:09.818534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:09.819237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.819773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.820482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.821235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:09.821544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:09.823082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:09.831763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 156/314] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:09.837181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:09.941636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b9dd6-5508-4329-a7bf-1791cbc222b1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:09.976859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:09.977735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:09.978021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:09.978301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] PATH_INFO: `/v3/users/ec834e28e5f040b5a7f7daf36c7db9fe` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:09.980178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] RBAC: Authorizing `identity:get_user(user_id=ec834e28e5f040b5a7f7daf36c7db9fe)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:09.981648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:09.983639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 159/315] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:09 2022] GET /identity/v3/users/ec834e28e5f040b5a7f7daf36c7db9fe => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:09.989034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:10.097120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06decb-c659-48eb-a493-12390d32b9d7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:10.132270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:10.132926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:10.133214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:10.133495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:10.180522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:10.204869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:10.206327 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:10.309800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 157/316] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:10.315275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:10.373665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:10.429839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7558de0-2b2b-4ca0-9132-005ec2f8e20d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:10.468678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c387e6-7dfe-4e3f-b492-fa21950f474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:10.469480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c387e6-7dfe-4e3f-b492-fa21950f474c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:10.469765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c387e6-7dfe-4e3f-b492-fa21950f474c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:10.470047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c387e6-7dfe-4e3f-b492-fa21950f474c None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:10.471705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c387e6-7dfe-4e3f-b492-fa21950f474c None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:10.473205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c387e6-7dfe-4e3f-b492-fa21950f474c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:10.475269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 160/317] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:10 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:10.481769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:10.562650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:10.591953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9a73-5c67-4971-9812-b321a3d05185 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:10.629089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:10.629752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:10.630038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:10.630316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:10.631182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] RBAC: Adding query filter params (role.id=881f45605c544dddb6830e6749387c9d, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800, user.id=ec834e28e5f040b5a7f7daf36c7db9fe) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:10.631498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:10.632943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b78271-4b90-4402-9ef5-2e69393ab751 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:10.639967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 158/318] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:10 2022] GET /identity/v3/role_assignments?user.id=ec834e28e5f040b5a7f7daf36c7db9fe&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=881f45605c544dddb6830e6749387c9d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:10.756192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cef126-43c7-4cf8-931d-a4375f94502e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:10.756383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cef126-43c7-4cf8-931d-a4375f94502e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:10.756677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cef126-43c7-4cf8-931d-a4375f94502e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:10.758369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 161/319] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:10.764475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:10.764770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:10.765053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:10.783519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:10.784747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:10.872050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 159/320] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:10.899330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cef126-43c7-4cf8-931d-a4375f94502e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.010449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cef126-43c7-4cf8-931d-a4375f94502e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:11.034567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:11.050366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caef709-a141-465f-bd7a-419ce459ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:11.051190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caef709-a141-465f-bd7a-419ce459ebc1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.051475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caef709-a141-465f-bd7a-419ce459ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.051751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caef709-a141-465f-bd7a-419ce459ebc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.052894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caef709-a141-465f-bd7a-419ce459ebc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:11.054370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caef709-a141-465f-bd7a-419ce459ebc1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:11.056187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 162/321] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:11.062011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.132116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:11.166730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc10ba-668b-444a-9c2c-8e310fdb4c50 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:11.201020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156172e-adbe-4164-81a2-4c4ee3f9daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:11.201684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156172e-adbe-4164-81a2-4c4ee3f9daac None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.201968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156172e-adbe-4164-81a2-4c4ee3f9daac None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.202245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156172e-adbe-4164-81a2-4c4ee3f9daac None admin] PATH_INFO: `/v3/groups` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.202966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156172e-adbe-4164-81a2-4c4ee3f9daac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:11.204421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156172e-adbe-4164-81a2-4c4ee3f9daac None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:11.222699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 160/322] 10.222.0.41 () {60 vars in 1134 bytes} [Mon Jun 6 11:50:11 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:11.240532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b83773-9ef0-41eb-9be5-04b5494abeef None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.240841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b83773-9ef0-41eb-9be5-04b5494abeef None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.241099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b83773-9ef0-41eb-9be5-04b5494abeef None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.242818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 163/323] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:11.248983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.249382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.249738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.269474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:11.270690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:11.349264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 161/324] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:11.372416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b83773-9ef0-41eb-9be5-04b5494abeef None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.484092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b83773-9ef0-41eb-9be5-04b5494abeef None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:11.521682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190f79-40f7-4a18-9d6e-19f00135dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:11.522759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190f79-40f7-4a18-9d6e-19f00135dfdd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.523208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190f79-40f7-4a18-9d6e-19f00135dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.523630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190f79-40f7-4a18-9d6e-19f00135dfdd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.536156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190f79-40f7-4a18-9d6e-19f00135dfdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-dd190f79-40f7-4a18-9d6e-19f00135dfdd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:11.544558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:11.545651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:11.546655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:11.548712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 164/325] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:11.555296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.617805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:11.622033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fddfe-ba0a-455b-98a3-006c5ff3c98d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.622324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fddfe-ba0a-455b-98a3-006c5ff3c98d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.622610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fddfe-ba0a-455b-98a3-006c5ff3c98d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.624301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 165/326] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:11.630826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.631153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.631443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.650003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:11.651918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:11.675900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b137-a59f-4ee8-ae02-d94d574ba3c4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:11.720417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:11.723656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:11.724909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.725400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.726069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.727127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:11.728227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:11.730631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:11.742246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 162/327] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:11.745350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 166/328] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:11.748952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.771677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.881324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67505d37-781a-433e-85f9-7af7d6e0df61 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:11.888569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7534c-7dfa-4479-8163-9dc099577559 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:11.907548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:11.922390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:11.923403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.923695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.923978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] PATH_INFO: `/v3/users/502fab24c7924de2bbb97cb051a616c0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.925312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] RBAC: Authorizing `identity:get_user(user_id=502fab24c7924de2bbb97cb051a616c0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:11.926772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:11.928863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 163/329] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:11 2022] GET /identity/v3/users/502fab24c7924de2bbb97cb051a616c0 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:11.930088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:11.936105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:11.936105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:11.936105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:11.936412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:11.951817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:11.964170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:11.965321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:11.966379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 167/330] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:11.971679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:12.017119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:12.054557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a87bb-a133-4e84-b9d9-c4abb3b9fb5e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:12.096338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:12.099033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeec707-640c-4950-8123-72259cbd9890 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:12.099391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:12.099818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:12.100245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] PATH_INFO: `/v3/system/users/502fab24c7924de2bbb97cb051a616c0/roles/881f45605c544dddb6830e6749387c9d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:12.101928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=502fab24c7924de2bbb97cb051a616c0, role_id=881f45605c544dddb6830e6749387c9d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:12.103522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:12.119669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 164/331] 10.222.0.41 () {58 vars in 1334 bytes} [Mon Jun 6 11:50:11 2022] PUT /identity/v3/system/users/502fab24c7924de2bbb97cb051a616c0/roles/881f45605c544dddb6830e6749387c9d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:12.145493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:12.146156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:12.146402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:12.146682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:12.147476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:12.147843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:12.149350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:12.160056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 168/332] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:12.165892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:12.250914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:12.312939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375c979-405c-4bd8-be25-c39d4207156b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:12.355929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abd8d7-3ee2-4862-9cd7-a6c8ac6f50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:12.357210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abd8d7-3ee2-4862-9cd7-a6c8ac6f50c6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:12.357708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abd8d7-3ee2-4862-9cd7-a6c8ac6f50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:12.358222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abd8d7-3ee2-4862-9cd7-a6c8ac6f50c6 None admin] PATH_INFO: `/v3/users/f730cc219f4247f98d0b291fa9774c4e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:12.359885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abd8d7-3ee2-4862-9cd7-a6c8ac6f50c6 None admin] RBAC: Authorizing `identity:get_user(user_id=f730cc219f4247f98d0b291fa9774c4e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:12.361786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abd8d7-3ee2-4862-9cd7-a6c8ac6f50c6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:12.366552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 165/333] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:12 2022] GET /identity/v3/users/f730cc219f4247f98d0b291fa9774c4e => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:12.370359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:12.483222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:12.486499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af545959-f977-4b75-80bc-746132c4bb7a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:12.526015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:12.526690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:12.527211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:12.527482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:12.528327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] RBAC: Adding query filter params (role.id=5a7ddb9a26de4c15abc5d71777d9e61b, user.id=f730cc219f4247f98d0b291fa9774c4e, scope.system=all) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:12.528647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:12.530129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbb439-96e3-4ec0-8751-bbbb8b011e8c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:12.543072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 169/334] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 11:50:12 2022] GET /identity/v3/role_assignments?user.id=f730cc219f4247f98d0b291fa9774c4e&scope.system=all&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:14.130284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9f362-73d5-481c-9ef4-de5be15d9a4a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.131367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9f362-73d5-481c-9ef4-de5be15d9a4a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.131868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9f362-73d5-481c-9ef4-de5be15d9a4a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.133462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 166/335] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:14.141427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aefc-b40e-4b2b-a67e-5dc73dac916f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.142341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aefc-b40e-4b2b-a67e-5dc73dac916f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.142827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aefc-b40e-4b2b-a67e-5dc73dac916f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.163412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0447aefc-b40e-4b2b-a67e-5dc73dac916f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:14.164950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0447aefc-b40e-4b2b-a67e-5dc73dac916f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:14.243317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 170/336] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:14.270344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9f362-73d5-481c-9ef4-de5be15d9a4a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:14.396818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4dff-f6ff-4b82-a364-058563039365 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.397458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4dff-f6ff-4b82-a364-058563039365 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.397945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4dff-f6ff-4b82-a364-058563039365 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.399579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 171/337] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:50:14.406188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.406777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.407374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.421705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9f362-73d5-481c-9ef4-de5be15d9a4a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:14.443454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:14.444917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:14.451216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:14.474097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:14.474964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.475252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.475532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.496112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:14.505378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:14.506527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:14.507349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:14.512216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 167/338] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:14.518314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:14.613195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 172/339] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:14.665165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:14.691231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507f541-2e3c-4372-93ff-a86b69050558 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:14.701212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:14.739140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:14.740249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.740681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.741298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.742867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:14.743678 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:14.748860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:14.760529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 168/340] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:14.766608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:14.819812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbdf3f-b25a-4bb5-ba1b-091c3c74bb49 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:14.828627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:14.867910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:14.868783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.869692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.870063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:14.883492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:14.893425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:14.894352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:14.895175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 173/341] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:14.902487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:14.924726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadcb55-e34d-4777-a05a-912651ec6af7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:14.980109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:14.981190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:14.981799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:14.982380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.006644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:15.042707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8648140-bb6c-4605-9784-fc7b135f2a92 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:15.088812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:15.089494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.089782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.090067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.090848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:15.091191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.098949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.102518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 174/342] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:15.108742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:15.144067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.145479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.232033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:15.297878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99313b-d4f7-4c9c-9a04-94b6c1b7bbfa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:15.351348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 169/343] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:15.352485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:15.357104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:15.357777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.358066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.358349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.374407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:15.471767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.473594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.475721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fed77-4e42-4ea1-bdbe-145bb56833e2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:15.511353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:15.512231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.512517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.512794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.514253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.516656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.522871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 170/344] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:15 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:15.525963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:15.559251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:15.638065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfa567-dc96-46c5-b773-cafecd3ff271 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:15.676621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:15.678110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.678110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.678431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.685864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.688668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.689657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 171/345] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:15 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:15.695460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:15.728118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 175/346] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 11:50:15.733757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:15.787280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:15.791954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:15.810687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012aff1-64ea-4b86-a156-fb9b8caa7f6d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:15.850660 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:15.851513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.851806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.852085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.852927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=6fced2c17e3741e992d8466df4bd6539) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:15.853850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.856487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.863730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 172/347] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:15 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=36fb427222cd413e808b7d4028e53faf => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:15.878379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1aa2e-827f-4a7d-8c4e-f5c2ab1d3650 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:15.920003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae03d-1892-496a-b0dc-565bdaab6db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:15.921260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae03d-1892-496a-b0dc-565bdaab6db9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.921745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae03d-1892-496a-b0dc-565bdaab6db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.922250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae03d-1892-496a-b0dc-565bdaab6db9 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.923837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ae03d-1892-496a-b0dc-565bdaab6db9 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:15.925588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ae03d-1892-496a-b0dc-565bdaab6db9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:15.927693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 176/348] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:15 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:15.933680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:15.994872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ac28-adc5-430e-8178-e227894b555c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:15.995253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ac28-adc5-430e-8178-e227894b555c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:15.995486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ac28-adc5-430e-8178-e227894b555c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:15.997062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 177/349] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:16.005019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e58e-082a-4381-b9b1-026bbafff6fc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.005340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e58e-082a-4381-b9b1-026bbafff6fc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.005585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1e58e-082a-4381-b9b1-026bbafff6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.052006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bde1e58e-082a-4381-b9b1-026bbafff6fc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:16.053570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bde1e58e-082a-4381-b9b1-026bbafff6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:16.077040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1a198-ef51-4b6c-af7a-556518aa22bf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.122573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e999d58-b95a-4b1c-9041-3755f47c952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:16.123581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e999d58-b95a-4b1c-9041-3755f47c952b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.124039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e999d58-b95a-4b1c-9041-3755f47c952b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.124483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e999d58-b95a-4b1c-9041-3755f47c952b None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.126006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e999d58-b95a-4b1c-9041-3755f47c952b None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:16.127739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e999d58-b95a-4b1c-9041-3755f47c952b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:16.129997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 173/350] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:15 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:16.131497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d964fae-98da-45ab-abaa-a3fc94c5c38c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.131947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d964fae-98da-45ab-abaa-a3fc94c5c38c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.132384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d964fae-98da-45ab-abaa-a3fc94c5c38c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.137777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 174/351] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:16.140904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d964fae-98da-45ab-abaa-a3fc94c5c38c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:16.155130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:16.211885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 178/352] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:16.219315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.219623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.220056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.256666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:16.261178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:16.321194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:16.347786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d964fae-98da-45ab-abaa-a3fc94c5c38c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.374991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 179/353] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:16.375904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:16.448354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:16.449027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.449318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.449599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.450441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=6fced2c17e3741e992d8466df4bd6539) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:16.450753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:16.452219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:16.469137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 175/354] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:16 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 260 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:16.470059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:16.502748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad9f5e-7487-4627-bb20-884009533816 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.545308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:16.546126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.546414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.546701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.559942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:16.568960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:16.569982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:16.570828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 180/355] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:16.576770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:16.615095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:16.683223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:16.707119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609b545-6e1e-409f-8043-8cc784dd82a2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.731286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451c80-f7e7-4279-9339-392f8cbca9ec None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.747651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:16.748342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.748632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.748915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.749681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:16.749991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:16.752604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:16.763033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 181/356] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:16.768622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:16.788321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:16.789144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.789432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.789707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.811416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:16.819185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:16.820142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:16.820992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:16.825710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 176/357] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:16.827011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:16.877315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:16.889088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98512248-3c9b-44a2-8f2b-5a8e5bd14d3b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.949776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9395db-bb88-4013-99bb-d0f56f9aee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:16.950669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9395db-bb88-4013-99bb-d0f56f9aee1e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.950976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9395db-bb88-4013-99bb-d0f56f9aee1e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.951260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9395db-bb88-4013-99bb-d0f56f9aee1e None admin] PATH_INFO: `/v3/users/502fab24c7924de2bbb97cb051a616c0` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.952600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9395db-bb88-4013-99bb-d0f56f9aee1e None admin] RBAC: Authorizing `identity:get_user(user_id=502fab24c7924de2bbb97cb051a616c0)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:16.953979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9395db-bb88-4013-99bb-d0f56f9aee1e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:16.956055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 182/358] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:50:16 2022] GET /identity/v3/users/502fab24c7924de2bbb97cb051a616c0 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:16.957312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46672d-bcab-413b-8d99-d2fc18739bbe None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.957659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46672d-bcab-413b-8d99-d2fc18739bbe None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.957947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46672d-bcab-413b-8d99-d2fc18739bbe None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.959422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 183/359] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:16.961279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4c21c-0a3f-43ea-96c5-d10f898a8a9c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.961603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4c21c-0a3f-43ea-96c5-d10f898a8a9c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.961849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4c21c-0a3f-43ea-96c5-d10f898a8a9c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.964029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 184/360] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:16.966505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:16.966937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:16.967567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:16.986799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ef94-decb-49ea-87ac-64f06b3de21d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:16.991335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:16.992587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:17.038318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.045414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.046689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.047367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.048052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.049491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:17.050256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:17.051911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7f1dc-908b-4d71-b0d2-2975b03b6c75 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:17.065546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 177/361] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:17.067185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.067682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.068160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.082531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 185/362] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:17.083677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.090827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:17.092326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:17.176344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 178/363] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:17.177967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.237283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.252147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfef79a-8b2b-40fb-9ec6-d66e85eb23a3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:17.289669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.290448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.290910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.291196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.292046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] RBAC: Adding query filter params (role.id=881f45605c544dddb6830e6749387c9d, user.id=502fab24c7924de2bbb97cb051a616c0, scope.system=all) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:17.292531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:17.293834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:17.312260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 186/364] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/role_assignments?user.id=502fab24c7924de2bbb97cb051a616c0&scope.system=all&role.id=881f45605c544dddb6830e6749387c9d => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:17.313110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.314810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56bb2-e371-4ee3-b416-cef2379e2b2f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:17.344377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.360576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.361446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.361884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.362368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.442688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:17.444439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:17.492812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a5270-46b5-4952-bfd0-8c8e3b04d3f6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:17.536223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.537040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.537329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.537618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.552494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:17.560693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:17.561639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:17.562425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 187/365] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:17.563131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.574718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.642416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 179/366] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:17.643328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.683604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea16cf-0263-4e3b-88b7-3906788ebfef None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:17.713232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.761487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.762317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.762605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.762986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.776164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:17.784639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:17.785568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 188/367] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:17.787480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.816434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb96e7-4f7b-4f77-89f7-6191efed669e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:17.855699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.856927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.857447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.857932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.859042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:17.859585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:17.861217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:17.863340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.872606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 180/368] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:17.873911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.906091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e71e8-83a7-43fe-86f2-32d896576020 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:17.947273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:17.948149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:17.948478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:17.948766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:17.949984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:17.951523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:17.953210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 189/369] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:17.953976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:17.973350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:17.990532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e08a07-b6a9-4642-aa52-7d98544eb611 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.033359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.034313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.034819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.035457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.036415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:18.036934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.038518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.048301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 181/370] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:18.049489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.073874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89556116-a83c-4588-bfd6-38b2dfbb595f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.111863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.112520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.112807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.113091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.131197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:18.168687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f86f7-ba6a-4099-b82a-52992c16c826 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.190335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.191590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.205891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.207444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.207944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.208463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.209865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.211573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.213687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 182/371] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:18.214879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.235065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:18.301674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 190/372] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:18.302333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.329720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace057-fae6-4f8c-8989-ebb643f25f36 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.367213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:18.369225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.370183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.370698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.371218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.411597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b6c6-9d45-430d-891b-0c29babca034 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.450201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.450720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.451084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.451378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.452263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:18.452370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.452646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.454058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.454523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.461177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 191/373] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:18.462039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.476212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:18.576684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 183/374] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:18.583437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.593404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07549b-365a-45bb-b7b7-2387c7d69891 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.623248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:18.639364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.640239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.640522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.640798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] PATH_INFO: `/v3/projects/a70232cae3264ff0a2f3b7ce99f87508` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.643086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] RBAC: Authorizing `identity:get_project(project_id=a70232cae3264ff0a2f3b7ce99f87508)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.644646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.646857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 192/375] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/projects/a70232cae3264ff0a2f3b7ce99f87508 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:18.656048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.719379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48557398-b183-4f71-92a1-a40c949958e6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.741190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:18.766970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.768834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.769617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.770123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.771675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.773557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.775991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 184/376] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:18.782911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.887400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf32fa-9b48-4bb0-bf55-32f0e228962f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.907672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48d7f-4ad4-47c9-97e0-67d109cdecb6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:18.950276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.951775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.952718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.953210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.954648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.956743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.959018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 185/377] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:18.966213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:18.967210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:18.968089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:18.968374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:18.968658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] PATH_INFO: `/v3/groups/e64d245861824c308ea7d4950776c75a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:18.983849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] RBAC: Authorizing `identity:get_group(group_id=e64d245861824c308ea7d4950776c75a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:18.985252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:18.987005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 193/378] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/groups/e64d245861824c308ea7d4950776c75a => generated 229 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:18.995219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:19.052882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:19.072121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:19.152696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc248f1-6445-41ce-b3a8-ed352694b8c3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:19.172832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff1039-8312-419c-9053-0dce80b437e1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:19.202266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:19.202966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.203251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.203803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.204650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:19.204970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:19.206946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04287d6e-8935-4f34-87ab-945105f938f1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:19.218342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 186/379] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=36fb427222cd413e808b7d4028e53faf => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:19.218342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f053678-e522-4483-a000-6e71b3d8f2f1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.218342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f053678-e522-4483-a000-6e71b3d8f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.218342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f053678-e522-4483-a000-6e71b3d8f2f1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.218342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 187/380] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:19.224155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.224473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.224700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.227613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:19.228548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.229141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.229632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.230790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] RBAC: Adding query filter params (group.id=e64d245861824c308ea7d4950776c75a, role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=a70232cae3264ff0a2f3b7ce99f87508) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:19.235447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:19.236941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:19.251220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 194/381] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:18 2022] GET /identity/v3/role_assignments?group.id=e64d245861824c308ea7d4950776c75a&scope.project.id=a70232cae3264ff0a2f3b7ce99f87508&role.id=b4418dc4009641b183fe42e56f1e484e => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:19.257728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:19.259125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:19.375060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 188/382] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:19.414268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:19.548114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:19.555179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dde6c-dc87-465d-a4c4-22c3f45a8398 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:19.596044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7493268-2874-43f7-b088-08f950f61c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:19.596912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493268-2874-43f7-b088-08f950f61c71 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.597204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493268-2874-43f7-b088-08f950f61c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.597483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493268-2874-43f7-b088-08f950f61c71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.612420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7493268-2874-43f7-b088-08f950f61c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d7493268-2874-43f7-b088-08f950f61c71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:19.621412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:19.622342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:19.624321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 195/383] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:19.635977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:19.708043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:19.729875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105a747-98e8-419d-a754-14ce28423784 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.730177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105a747-98e8-419d-a754-14ce28423784 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.730443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105a747-98e8-419d-a754-14ce28423784 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.732017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 196/384] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:19.737456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.737753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.738030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.758402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:19.760673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:19.830743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384afaea-fcc7-4d94-bf76-ac41fff690a7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:19.838938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 197/385] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:19.864201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:19.913589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:19.914334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:19.914625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:19.915444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:19.916208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:19.916515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:19.917909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:19.934243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 189/386] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:19.939696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:19.958056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:19.977935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bafbe-fa9a-4091-9615-f8c32661852c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.017799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.018665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.018987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.019273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.031714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:20.039763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:20.040739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:20.041549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:20.042659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 198/387] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:20.053084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:20.176749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc0cc3-4d8f-4702-a85c-4a54e654013b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.206666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2aeeb-0a03-49d3-9e90-100a4475ae0f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.225002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:20.232042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.232915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.233638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.233638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.237370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.238975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.241343 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 190/388] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:19 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:20.247409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:20.250595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.251685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.252190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.252680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.253633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] RBAC: Adding query filter params (name=member) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:20.254143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.255990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.265906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 199/389] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:20.271084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:20.286418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:20.385495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c6fa-1a43-4723-a3da-16f4c03cc612 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.423824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.425076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.425567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.426044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.427519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.429311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.431708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 200/390] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:20 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:20.444415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:20.480898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4020a1-7b6a-48af-8915-a2e1ec7e823f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.579625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:20.583622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bd249-d693-4484-964a-961955d5c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.584512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bd249-d693-4484-964a-961955d5c55e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.584796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bd249-d693-4484-964a-961955d5c55e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.585072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bd249-d693-4484-964a-961955d5c55e None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.586347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bd249-d693-4484-964a-961955d5c55e None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.587940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bd249-d693-4484-964a-961955d5c55e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.592248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 191/391] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:20 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:20.595497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bd249-d693-4484-964a-961955d5c55e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:20.615571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb26ede-7c97-4d10-b7ed-d1737f3b9ec8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.660278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.661561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.662070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.662559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.663990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.668083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.670580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 201/392] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:20 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:20.675707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:20.707716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:20.784746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bd249-d693-4484-964a-961955d5c55e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.791789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d3194-abaf-435e-b46f-d93d4f71d5f4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:20.829559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1147b16-d5e8-47a8-80d4-c3e342596a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.830957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1147b16-d5e8-47a8-80d4-c3e342596a27 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.831564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1147b16-d5e8-47a8-80d4-c3e342596a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.832067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1147b16-d5e8-47a8-80d4-c3e342596a27 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539/groups/5bdd979ed8134ada8d20580bc2580149/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.834299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1147b16-d5e8-47a8-80d4-c3e342596a27 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fced2c17e3741e992d8466df4bd6539, group_id=5bdd979ed8134ada8d20580bc2580149, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.837317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1147b16-d5e8-47a8-80d4-c3e342596a27 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.844585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a1185-9791-4efb-826c-e6122e91ea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:20.845368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a1185-9791-4efb-826c-e6122e91ea43 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:20.845651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a1185-9791-4efb-826c-e6122e91ea43 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:20.845931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a1185-9791-4efb-826c-e6122e91ea43 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539/groups/5bdd979ed8134ada8d20580bc2580149/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:20.849833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a1185-9791-4efb-826c-e6122e91ea43 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fced2c17e3741e992d8466df4bd6539, group_id=5bdd979ed8134ada8d20580bc2580149, role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:20.851333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a1185-9791-4efb-826c-e6122e91ea43 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:20.857176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 202/393] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 11:50:20 2022] PUT /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539/groups/5bdd979ed8134ada8d20580bc2580149/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 11:50:20.876382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 192/394] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 11:50:20 2022] PUT /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539/groups/5bdd979ed8134ada8d20580bc2580149/roles/36fb427222cd413e808b7d4028e53faf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 11:50:21.370152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d4bb9-26d8-4f23-afc2-7b57bcf06f3c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:21.370459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d4bb9-26d8-4f23-afc2-7b57bcf06f3c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:21.370962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d4bb9-26d8-4f23-afc2-7b57bcf06f3c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:21.372695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 203/395] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:21.378314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:21.378751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:21.378960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:21.403623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:21.404827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:21.488417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 193/396] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:21.526748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d4bb9-26d8-4f23-afc2-7b57bcf06f3c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:21.529766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:21.654893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d4bb9-26d8-4f23-afc2-7b57bcf06f3c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:21.696649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:21.697457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:21.697743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:21.698020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:21.710505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:21.719242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:21.720395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:21.721670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 204/397] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:21.727316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:21.765678 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:21.850056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844704df-17cc-4036-b534-8c23e7d8add6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:21.892326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:21.893004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:21.893287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:21.893567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:21.894319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:21.894629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:21.896057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:21.905403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 194/398] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:21.910984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:21.985340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:22.034663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237801d-7b38-4c67-abe5-25e44ba01849 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:22.077147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8cb48-e23b-4163-820e-c1f5e4a7006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:22.078077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8cb48-e23b-4163-820e-c1f5e4a7006e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.078365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8cb48-e23b-4163-820e-c1f5e4a7006e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.078706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8cb48-e23b-4163-820e-c1f5e4a7006e None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.079986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8cb48-e23b-4163-820e-c1f5e4a7006e None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:22.081459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8cb48-e23b-4163-820e-c1f5e4a7006e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:22.083416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 205/399] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:21 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:22.089293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.200950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:22.210185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2310d8-9a0f-4962-8e7b-432d06be210e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:22.212349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439bea7-1e13-4d63-a332-6b080693e7e9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.212724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439bea7-1e13-4d63-a332-6b080693e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.213007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439bea7-1e13-4d63-a332-6b080693e7e9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.214583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 206/400] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:22.220046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a515b-d6ee-4892-8701-5b33cd0c3d4f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.220337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a515b-d6ee-4892-8701-5b33cd0c3d4f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.220617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a515b-d6ee-4892-8701-5b33cd0c3d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.251911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-da7a515b-d6ee-4892-8701-5b33cd0c3d4f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:22.251911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-da7a515b-d6ee-4892-8701-5b33cd0c3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:22.264113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:22.265411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.265889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.266374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.271697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:22.273397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:22.276360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 195/401] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:22 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:22.282287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.363902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 207/402] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:22.371059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6adf-b4d5-4016-98f2-eafd9af1442e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.371059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6adf-b4d5-4016-98f2-eafd9af1442e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.371059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6adf-b4d5-4016-98f2-eafd9af1442e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.378122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 208/403] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:22.379632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2146473-ae73-4fae-9694-478c4317180d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.380737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2146473-ae73-4fae-9694-478c4317180d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.381344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2146473-ae73-4fae-9694-478c4317180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.410917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b2146473-ae73-4fae-9694-478c4317180d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:22.413180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b2146473-ae73-4fae-9694-478c4317180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:22.434968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05397d47-29db-4d4f-b989-8ef032246a97 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:22.482344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:22.483551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.484020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.484479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/groups/5bdd979ed8134ada8d20580bc2580149/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.486701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec71c3cb2fa34334b0a06778fa4f9800, group_id=5bdd979ed8134ada8d20580bc2580149, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:22.488380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:22.489525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:22.508072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 196/405] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 11:50:22 2022] PUT /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/groups/5bdd979ed8134ada8d20580bc2580149/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 11:50:22.509384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.510937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 209/405] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:22.530579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2146473-ae73-4fae-9694-478c4317180d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.551592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:22.658960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d185e49-be38-4fe0-9631-ae7393987f67 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:22.682406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2146473-ae73-4fae-9694-478c4317180d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:22.703309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:22.704526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.705096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.705582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.720015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:22.727860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:22.728861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.729337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.729769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:22.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:22.733230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:22.734083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:22.738982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 197/406] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:22.739904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.748659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:22.760998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:22.762136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:22.763134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:22.765885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 210/407] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:22.777522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.843457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:22.854606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:22.867189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0ad64-8e5f-4e1f-98fd-021b2d499cb8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:22.927573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:22.928250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:22.929250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:22.929356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:22.930126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:22.930440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:22.932352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:22.945315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 198/408] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:22.951809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:22.955158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477ca5a-b66f-49c3-aadd-54431431f00d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.025323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.025990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.026276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.026553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.027339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:23.027701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.029072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.053343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 211/409] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:23.059373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:23.075232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecf12e-9629-4b9a-bfed-ee39f486a51d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.117327 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.119871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.120399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.120880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] PATH_INFO: `/v3/groups/e64d245861824c308ea7d4950776c75a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.122343 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] RBAC: Authorizing `identity:get_group(group_id=e64d245861824c308ea7d4950776c75a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.124255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.126592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 199/410] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:22 2022] GET /identity/v3/groups/e64d245861824c308ea7d4950776c75a => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:23.145765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:23.172267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:23.217708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e612-b516-4e01-87f0-0f40a418cc32 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.232065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:23.255569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.256422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.256713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.256992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.258172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.259779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.261824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 212/411] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:23 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:23.267215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:23.345968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6591f4-12f2-4c5f-bfdb-02d7ce328017 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.376247 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347b104-d897-4639-89a3-aa991974b854 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.420209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.421103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.421391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.421669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] PATH_INFO: `/v3/projects/a70232cae3264ff0a2f3b7ce99f87508` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.423569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=a70232cae3264ff0a2f3b7ce99f87508)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.425056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.428042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.428923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.429203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.429479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.431712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.433171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.435522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 200/412] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:23 2022] GET /identity/v3/projects/a70232cae3264ff0a2f3b7ce99f87508 => generated 344 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:23.443221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 213/413] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:23 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:23.447547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:23.448773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:23.535652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:23.595708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66df08-de80-4c64-ad7e-3cde195e87c3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.630233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:23.652676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0d956-60de-44d1-9dd1-0094e1122ad5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:23.692636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09267898-3e24-4d9c-bb8b-407c13c90a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.693518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267898-3e24-4d9c-bb8b-407c13c90a93 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.693799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267898-3e24-4d9c-bb8b-407c13c90a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.694072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267898-3e24-4d9c-bb8b-407c13c90a93 None admin] PATH_INFO: `/v3/projects/a70232cae3264ff0a2f3b7ce99f87508/groups/e64d245861824c308ea7d4950776c75a/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.697167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09267898-3e24-4d9c-bb8b-407c13c90a93 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70232cae3264ff0a2f3b7ce99f87508, group_id=e64d245861824c308ea7d4950776c75a, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.698630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09267898-3e24-4d9c-bb8b-407c13c90a93 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.701882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0e5cd-0d14-4bdc-b6e3-f383a70ab992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:23.702765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0e5cd-0d14-4bdc-b6e3-f383a70ab992 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.703073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0e5cd-0d14-4bdc-b6e3-f383a70ab992 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.703350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0e5cd-0d14-4bdc-b6e3-f383a70ab992 None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/groups/5bdd979ed8134ada8d20580bc2580149/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.706471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0e5cd-0d14-4bdc-b6e3-f383a70ab992 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec71c3cb2fa34334b0a06778fa4f9800, group_id=5bdd979ed8134ada8d20580bc2580149, role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:23.707948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0e5cd-0d14-4bdc-b6e3-f383a70ab992 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:23.731507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 214/415] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 11:50:23 2022] PUT /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800/groups/5bdd979ed8134ada8d20580bc2580149/roles/36fb427222cd413e808b7d4028e53faf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:23.731841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 201/415] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 11:50:23 2022] PUT /identity/v3/projects/a70232cae3264ff0a2f3b7ce99f87508/groups/e64d245861824c308ea7d4950776c75a/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:23.755841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6eeb7-55e0-4721-833c-32dce10ff431 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.756492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6eeb7-55e0-4721-833c-32dce10ff431 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.756980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6eeb7-55e0-4721-833c-32dce10ff431 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.759177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 215/416] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:23.765471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:23.766096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:23.766610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:23.795393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:23.797623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:23.934737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 202/417] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:23.974413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6eeb7-55e0-4721-833c-32dce10ff431 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:24.117117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6eeb7-55e0-4721-833c-32dce10ff431 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:24.130186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:24.165277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:24.166121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:24.166414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:24.166700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:24.186702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:24.195111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:24.196748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:24.197616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:24.198229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 216/418] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:24.204636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:24.303010 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:24.354419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ced106-442d-499d-aed6-e5dd54ef6ee9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:24.394304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:24.394930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:24.395355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:24.395636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:24.396392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:24.396706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:24.398086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e04c6-bcfb-4e8b-8998-36b6d14d4e05 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:24.407130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 203/419] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:24.412696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:24.513504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b242b1-ee6f-4474-986c-66c4c2a1fb36 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:24.514019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b242b1-ee6f-4474-986c-66c4c2a1fb36 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:24.514111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b242b1-ee6f-4474-986c-66c4c2a1fb36 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:24.515719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 204/420] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:24.525094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:24.525380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:24.525671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:24.542727 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc466540-1d39-4e11-b87a-56fabccc1c39 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:24.564033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:24.566013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:24.594595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:24.595639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:24.596154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:24.596665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:24.654237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 205/421] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:24.682865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:24.701077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:24.710656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:24.713428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:24.811847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:24.823352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 217/422] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:24.833851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:24.908876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e57c0-d461-492c-8e62-0da7fae44b23 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:24.959768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:24.980292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:24.981105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:24.981394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:24.981674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:24.999929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:25.023178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:25.024726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:25.025579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:25.026329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 206/423] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:25.032774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.033408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5662ac-5487-4f7a-9113-8d805c03c51a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.112573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.113426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.113713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.113993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.119614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:25.121117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:25.122867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 218/424] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:24 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:25.128259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.153225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4b40f-f8b4-4c08-9ebb-acc96f005a3a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.203238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:25.227310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.230196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.230942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.233054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.234276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:25.235673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:25.237321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529c598-8399-40dc-956e-cedb0f433486 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:25.267312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 207/425] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:25.269855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff3302-49c1-40bb-a5bb-1ec335bc8473 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.270301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff3302-49c1-40bb-a5bb-1ec335bc8473 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.270727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff3302-49c1-40bb-a5bb-1ec335bc8473 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.272399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 208/426] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:25.273619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff3302-49c1-40bb-a5bb-1ec335bc8473 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.331657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860eca-618f-4483-aed3-b13fc9b755ad None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.375358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed2c41-eb34-455a-b153-349400853d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.376368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2c41-eb34-455a-b153-349400853d33 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.377124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2c41-eb34-455a-b153-349400853d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.377516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2c41-eb34-455a-b153-349400853d33 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.379230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed2c41-eb34-455a-b153-349400853d33 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:25.381396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed2c41-eb34-455a-b153-349400853d33 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:25.384789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 219/427] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:25.386336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.387044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.387492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.395025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:25.405604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff3302-49c1-40bb-a5bb-1ec335bc8473 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.410078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:25.411425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:25.447629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.448648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.449079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.449800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.490462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 220/428] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:25.495869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.516335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:25.535555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:25.537620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:25.626518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa59f4-efaa-4e5b-abcb-c7cd0b398dcd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.667942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.668951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.669435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.669962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.671142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=6fced2c17e3741e992d8466df4bd6539) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:25.671674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:25.673532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:25.675844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:25.682017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 209/429] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:25.682292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 221/430] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=36fb427222cd413e808b7d4028e53faf => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:25.688935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.697111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.801727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:25.821804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace0c92-9009-4e7d-8aef-be66360d04b1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.866335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.867439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.867830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.868415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.882140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:25.892916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:25.893879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:25.894688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 210/431] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:25.915262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:25.938273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035dcc8-5d1a-40da-a1e6-62a78181948e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:25.979435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:25.980464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:25.980857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:25.981238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] PATH_INFO: `/v3/projects/6fced2c17e3741e992d8466df4bd6539` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:25.982536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] RBAC: Authorizing `identity:get_project(project_id=6fced2c17e3741e992d8466df4bd6539)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:25.984148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:25.986312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 222/432] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/projects/6fced2c17e3741e992d8466df4bd6539 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:25.995184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.065816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7d936-f4e1-47c5-8fbb-5d3b235cb8cf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.099978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.108054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.108731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.109065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.109351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.110105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:26.110416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.111907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.126656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 211/433] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:26.127662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.140790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.148495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e639-e4ba-4342-bd05-a32612706d98 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.201240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.202145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.202425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.202699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.204001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.205470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.208717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 223/434] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:25 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:26.218974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.249714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8347a3-caf3-43da-bda2-702db8c4c1d8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.286250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.286968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.287257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.287537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.337246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9107-c541-4ced-804a-004221c597b9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.364981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.366462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.374937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.375678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.375970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.376248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.377125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=6fced2c17e3741e992d8466df4bd6539) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:26.377391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.377792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.378946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570650-8557-4518-96cd-60ccb3322468 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.381314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.388098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 224/435] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=6fced2c17e3741e992d8466df4bd6539&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:26.389386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b9bb0-120d-44bb-bfd0-9dfb95a5e9e7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.389677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b9bb0-120d-44bb-bfd0-9dfb95a5e9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.389956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b9bb0-120d-44bb-bfd0-9dfb95a5e9e7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.391467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 225/436] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:26.397047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.397344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.397603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.416284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:26.417499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:26.503253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 226/437] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:26.504138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 212/438] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:26.510259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.528000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.635165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc6a20-3cf6-42f8-b4a6-932b7e4bb722 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.671221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.673296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709a23-f0b2-4f1a-8c0e-d55a88b42b25 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.682291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.683662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.684627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.684791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.685528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.688300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.690209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.692961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 227/439] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:26.698741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.716650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.717462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.717747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.718026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.731870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:26.740910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:26.741818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:26.743297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 213/440] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:26.752030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.819260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327bacc-7593-42fd-8a1a-57b93e84790f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.865527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.866208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.866485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.866757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.869411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.871025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.872846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 228/441] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:26.878546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.891202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4849-feed-4965-84f9-500036ab3097 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:26.937723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:26.938372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:26.938661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:26.939309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:26.939741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:26.942483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:26.942483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:26.954529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 214/442] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:26.960130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:26.973520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:26.998578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.024461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d6669-8474-40ed-a22e-2b27efc8b613 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.066275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.066973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.067263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.067545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.068395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=5a7ddb9a26de4c15abc5d71777d9e61b, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:27.068710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.070720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f116c29-e46a-47b9-975f-df00e39d4286 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.077036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 229/443] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=5a7ddb9a26de4c15abc5d71777d9e61b => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:27.078562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a31f8-9d8a-49e3-b75b-4d132efdbd2a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.078911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a31f8-9d8a-49e3-b75b-4d132efdbd2a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.079179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a31f8-9d8a-49e3-b75b-4d132efdbd2a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.083282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07a87-a658-4b96-a680-115e469ee3bb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.084833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 230/444] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:27.087190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.087190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.087190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.108610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:27.108804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:27.130253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.133090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.133232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.133465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.203531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 231/445] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:27.230347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:27.233665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.235197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.269961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.296224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.345110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d97048-279a-48f6-ae88-d1b6ee04767a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.359810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 215/446] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:27.365842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:27.390934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.391741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.392024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.392317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.406661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:27.414729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:27.415684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:27.416477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 232/447] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:27.424779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:27.477452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5d4ca-aa99-43c3-8aee-7037edf5f79b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.510609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.516911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.517772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.518062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.518345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] PATH_INFO: `/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.519543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] RBAC: Authorizing `identity:get_project(project_id=ec71c3cb2fa34334b0a06778fa4f9800)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.521018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.522707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 216/448] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/projects/ec71c3cb2fa34334b0a06778fa4f9800 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:27.528206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:27.539601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bfb26-4d10-4973-8d21-029371087f1f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.556946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.574290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.574989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.575277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.575555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.576299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:27.576610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.578012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.586059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 233/449] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:27.591524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:27.629156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b1eb6-cdd5-4419-acbd-f7a77c33360c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.662177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.663056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.663342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.663621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] PATH_INFO: `/v3/groups/5bdd979ed8134ada8d20580bc2580149` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.664981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] RBAC: Authorizing `identity:get_group(group_id=5bdd979ed8134ada8d20580bc2580149)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.666445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.668100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 217/450] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/groups/5bdd979ed8134ada8d20580bc2580149 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:27.673725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:27.692406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95852986-4774-4a9f-b34b-e03f795fe9fb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.727713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.728288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.728574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.728863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.738316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.772241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715bde-520e-4a0a-850d-9afe37999c9f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:27.776661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:27.796422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.797896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.808260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:27.808911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:27.809199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:27.809480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:27.810300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] RBAC: Adding query filter params (group.id=5bdd979ed8134ada8d20580bc2580149, role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:27.810611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:27.812066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:27.818273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 218/451] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/role_assignments?group.id=5bdd979ed8134ada8d20580bc2580149&scope.project.id=ec71c3cb2fa34334b0a06778fa4f9800&role.id=36fb427222cd413e808b7d4028e53faf => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:27.901959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 234/452] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 11:50:27.907714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:28.029407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3a85a-07b9-40f4-88a5-ba75933d818c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:28.058019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:28.064103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:28.064970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:28.065257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:28.065536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] PATH_INFO: `/v3/projects/a70232cae3264ff0a2f3b7ce99f87508` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:28.066720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] RBAC: Authorizing `identity:get_project(project_id=a70232cae3264ff0a2f3b7ce99f87508)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:28.068201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:28.069884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 219/453] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:50:27 2022] GET /identity/v3/projects/a70232cae3264ff0a2f3b7ce99f87508 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:28.076144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:28.166438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:28.188158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d71ae3-eeb8-4582-94ae-3d53dce27e32 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:28.223832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bbbfa-a886-4cbb-840b-c075db6030ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:28.224681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bbbfa-a886-4cbb-840b-c075db6030ed None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:28.224992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bbbfa-a886-4cbb-840b-c075db6030ed None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:28.225271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bbbfa-a886-4cbb-840b-c075db6030ed None admin] PATH_INFO: `/v3/groups/e64d245861824c308ea7d4950776c75a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:28.226466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bbbfa-a886-4cbb-840b-c075db6030ed None admin] RBAC: Authorizing `identity:get_group(group_id=e64d245861824c308ea7d4950776c75a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:28.228292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bbbfa-a886-4cbb-840b-c075db6030ed None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:28.229898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 235/454] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 11:50:28 2022] GET /identity/v3/groups/e64d245861824c308ea7d4950776c75a => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:28.235693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:28.334739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c210-7ca8-4170-bdd5-02d0e60bb66b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:28.367762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:28.368407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:28.368689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:28.368968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:28.369781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] RBAC: Adding query filter params (group.id=e64d245861824c308ea7d4950776c75a, role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=a70232cae3264ff0a2f3b7ce99f87508) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:28.370094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:28.371543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaf6a0-b5bb-43b4-8cf7-cb8991dd713a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:28.377172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 220/455] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 11:50:28 2022] GET /identity/v3/role_assignments?group.id=e64d245861824c308ea7d4950776c75a&scope.project.id=a70232cae3264ff0a2f3b7ce99f87508&role.id=b4418dc4009641b183fe42e56f1e484e => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:31.349368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7bb8-1b85-4fd9-a3c4-f061738cc90e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.349368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7bb8-1b85-4fd9-a3c4-f061738cc90e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.349368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb7bb8-1b85-4fd9-a3c4-f061738cc90e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.349368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 236/456] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:31.354630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.355476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.355955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.377014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:31.378541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:31.450376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:31.455807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5ad5f-0a13-45ab-ac1c-c70aea1d29d7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.456433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5ad5f-0a13-45ab-ac1c-c70aea1d29d7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.456911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5ad5f-0a13-45ab-ac1c-c70aea1d29d7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.458702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 237/457] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:31.465021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df69e-d4ca-4d3f-8538-b4529e1fa301 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.465725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df69e-d4ca-4d3f-8538-b4529e1fa301 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.466207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df69e-d4ca-4d3f-8538-b4529e1fa301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.466658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 221/458] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:31.492055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:31.507873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-937df69e-d4ca-4d3f-8538-b4529e1fa301 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:31.509384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-937df69e-d4ca-4d3f-8538-b4529e1fa301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:31.591005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 238/459] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:31.592701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb44f9-4dcb-44a7-93ae-ea6a7b42dee1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.593167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb44f9-4dcb-44a7-93ae-ea6a7b42dee1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.593592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb44f9-4dcb-44a7-93ae-ea6a7b42dee1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.599320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 239/460] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:31.601419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c13f9-1777-482d-98aa-4e01aeb32cc3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:31.601682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.602146 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.602621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.619658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:31.621120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:31.639686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:31.640622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.640910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.641187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.654309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:31.656053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:31.680950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:31.681866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:31.682704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:31.682704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:31.682704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:31.682704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:31.684812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 222/461] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:31.685887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:31.714862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 240/462] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:31.714862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:31.721622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:31.762905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:31.824428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2221a9-4c64-4fbe-ae9a-c8b1016bd287 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:31.837676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7653e-a146-4aa1-9367-a65dc4446e97 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:31.868264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:31.869318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.869802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.870277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.871238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:31.871760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:31.873344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:31.884149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:31.884956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:31.885244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:31.885525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:31.886937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 241/463] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:31.888081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:31.898032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:31.899638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:31.923735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:31.924618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:31.925894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:31.925894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:31.925894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:31.925894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:31.925894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 223/464] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:31.925894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:31.966613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.004154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b465438-5bb1-4761-85ad-053b4f2d5186 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.044232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.045030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.045308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.045774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.047981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.054326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f75a9-b839-4679-a834-87f1b188710e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.057918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.059429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:32.076908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:32.077779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:32.078798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:32.078798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:32.078798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:32.078798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:32.078798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 242/465] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:32.079355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.097696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9aedbe-2608-430b-9716-5ea3f079ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.098762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9aedbe-2608-430b-9716-5ea3f079ab0c None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.099232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9aedbe-2608-430b-9716-5ea3f079ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.099516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9aedbe-2608-430b-9716-5ea3f079ab0c None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.100234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9aedbe-2608-430b-9716-5ea3f079ab0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.102236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9aedbe-2608-430b-9716-5ea3f079ab0c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.130962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 224/466] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:31 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:32.130962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16d28-ec8d-482d-bed2-b10399203a72 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.130962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16d28-ec8d-482d-bed2-b10399203a72 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.130962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16d28-ec8d-482d-bed2-b10399203a72 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.130962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 225/467] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:32.130962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f16d28-ec8d-482d-bed2-b10399203a72 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.206952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb86b7-2291-4094-ae41-d505035f5811 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.231904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.250641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.251436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.251788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.252021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.252777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:32.253148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.253411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f16d28-ec8d-482d-bed2-b10399203a72 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.254625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef7cf7-ccba-47a0-ad34-89acf79350f3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.277772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 243/468] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:32.277772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.277772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.277772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.292710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.293569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.293676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.293964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.294742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:32.295099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.296517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c818a17-6262-44eb-b406-02d218bb5e09 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.297918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:32.299187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:32.321177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 226/469] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:32.321177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aac6b-fc87-4a10-8f1c-0ed882755cf8 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.321177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aac6b-fc87-4a10-8f1c-0ed882755cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.321177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aac6b-fc87-4a10-8f1c-0ed882755cf8 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.321177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 227/470] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:32.321177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aac6b-fc87-4a10-8f1c-0ed882755cf8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.339718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.400399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 244/471] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:32.401241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.454584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aac6b-fc87-4a10-8f1c-0ed882755cf8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.496427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d24d-8e26-425e-a4fe-c1f0ab19e094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.497107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d24d-8e26-425e-a4fe-c1f0ab19e094 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.497396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d24d-8e26-425e-a4fe-c1f0ab19e094 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.497678 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d24d-8e26-425e-a4fe-c1f0ab19e094 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.498391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d24d-8e26-425e-a4fe-c1f0ab19e094 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.500611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5d24d-8e26-425e-a4fe-c1f0ab19e094 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.510495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713e84-9482-4418-9e44-6ef3ad7f2b1f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.522227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.524866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 228/472] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:32 2022] POST /identity/v3/users => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:32.526330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.526804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.527286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.548788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:32.551168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:32.559628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.560461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.560813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.561160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.562017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.563824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.590661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 245/473] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:32 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:32.591665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.650081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.660877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 229/474] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:32.703047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.807414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7a58a-79a8-4b0a-a943-c212e7cffa1f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.852530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.853505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.853933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.854269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.866867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ab75d-d9b2-4bfa-9dc0-a96c1f6ab7dc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:32.868647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.870139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:32.888374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:32.889300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:32.890398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:32.890398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:32.890398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:32.890398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:32.890398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 246/475] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:32.895654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:32.909074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:32.909746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:32.910043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:32.910306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:32.911137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:32.913104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:32.923410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.sql.core [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 11:50:32.923410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 11:50:32.923410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [parameters: {'id': 'cf22cafdb90b414787d18e7f2c139855', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 11:50:32.923410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 11:50:32.925972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 06 11:50:32.943813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return ref.to_dict() Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 11:50:32.944825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application yield resource Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise self.value Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application session.commit() Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 11:50:32.945911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.flush() Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application compat.raise_( Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 11:50:32.946870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 11:50:32.948484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 11:50:32.950576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [parameters: {'id': 'cf22cafdb90b414787d18e7f2c139855', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:32.954187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 11:50:32.956296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:32.958362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 230/476] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 11:50:32 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:32.969193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:33.043953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:33.046897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ca847-1c8b-4590-b0d7-22b6c2f6b0c4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:33.095080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2c5d-e506-4dad-8054-47312af9a50b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:33.096073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:33.097142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:33.098418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:33.098939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:33.099898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:33.100440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:33.102580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:33.119333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 247/477] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:33.126261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:33.148881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:33.149707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:33.149992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:33.150268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:33.173541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:33.184518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:33.185442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:33.186665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 231/478] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:50:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:33.196809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:33.287102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:33.322440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7464b-ce05-4503-93b3-55c57d3ad27c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:33.369747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053ac53-4c87-480b-ba93-0271dc6c91e2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:33.392174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402135bb-0cef-4572-9d15-7a4372ded6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:33.392856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402135bb-0cef-4572-9d15-7a4372ded6b0 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:33.393195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402135bb-0cef-4572-9d15-7a4372ded6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:33.393477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402135bb-0cef-4572-9d15-7a4372ded6b0 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:33.394185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402135bb-0cef-4572-9d15-7a4372ded6b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:33.395510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:33.395673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402135bb-0cef-4572-9d15-7a4372ded6b0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:33.409054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:33.409949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:33.410377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:33.410795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:33.411756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:33.412206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:33.414689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94439-5852-41fb-9bf6-e70c7e5fb253 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:33.424531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 232/479] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:33.441945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 248/480] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:33 2022] POST /identity/v3/users => generated 249 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:34.717481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a00aa-de19-4d61-90fe-69165f47feb0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:34.718032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a00aa-de19-4d61-90fe-69165f47feb0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:34.718032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a00aa-de19-4d61-90fe-69165f47feb0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:34.720050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 233/481] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:34.725451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:34.726621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:34.726621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:34.750212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:34.751448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:34.827062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 249/482] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:34.872936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a00aa-de19-4d61-90fe-69165f47feb0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:35.031671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a00aa-de19-4d61-90fe-69165f47feb0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:35.086025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:35.086871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.087159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.087440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.107532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:35.115148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:35.116018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:35.117933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 234/483] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:35.123063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:35.228355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:35.284511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad7d6-1ec3-4b86-bf31-923d2dd694af None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:35.338754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:35.339459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.339745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.340025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.340777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:35.341086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:35.342488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6ee81-e694-4c34-8427-b62cc53df616 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:35.354826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 250/484] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:35.360713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:35.419282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:35.488575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a13363-1d28-4f6a-b264-f66fb294d6de None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:35.521839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c0e14-896d-4de9-810d-4cc0e5b23016 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.522169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c0e14-896d-4de9-810d-4cc0e5b23016 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.522505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c0e14-896d-4de9-810d-4cc0e5b23016 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.524350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 251/485] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:35.536484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:35.537286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.537580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.537869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.544269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df38c2-5056-44b4-b3fa-4e36bd5dce26 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.544564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df38c2-5056-44b4-b3fa-4e36bd5dce26 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.544845 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df38c2-5056-44b4-b3fa-4e36bd5dce26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.551008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:35.552506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:35.570799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:35.571735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:35.572568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:35.572568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:35.572568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:35.572568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:35.576686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 235/486] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:35.577771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f6df38c2-5056-44b4-b3fa-4e36bd5dce26 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:35.577879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:35.579016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f6df38c2-5056-44b4-b3fa-4e36bd5dce26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:35.724104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 252/487] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:35.726210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827228ed-0e00-44f2-b5f2-7f962b3fd6d7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.727065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827228ed-0e00-44f2-b5f2-7f962b3fd6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.727065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827228ed-0e00-44f2-b5f2-7f962b3fd6d7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.732744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 253/488] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:35.734365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:35.736332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.736770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.737401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.747303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aca72-1b03-4bc7-848a-4506f54f74a3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:35.757860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:35.759622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:35.803671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:35.804855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:35.805288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:35.805727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:35.806697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:35.807339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:35.808987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:35.823051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 236/489] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:35.828543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:35.832106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:35.862883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 254/490] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:35.862883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:35.969527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b754e-f682-4fd2-b8ee-39c21d235941 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:35.994863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a87b63-f07c-4f8c-9372-ac1b2aa6a205 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.019807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.020905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.021399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.021822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.034405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:36.039818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.040925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.041441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.041923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:36.042522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:36.043421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:36.044234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:36.049181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 237/491] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:36.049979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.056952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.058829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.072811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:36.085178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:36.086327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:50:36.087244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:36.090821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 255/492] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:36.091940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.154331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.184917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551885f-9222-4bd0-8a42-399a93afedc6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.213023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7d112-e295-42c9-ba2b-67adb29e4eb1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.231464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.233003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.233502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.233987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.248543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:36.250398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.251172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.251461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.251745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.252501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:36.252812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.254204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:36.259311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:36.260217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:36.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:36.264534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 238/493] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:36.265524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 256/494] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:36.265832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.275997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.362756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.389725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2267-601d-4320-a893-d277f8f2fabd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.424904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.425574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.425867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.426152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.427028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:36.427346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.429031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.466346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 239/495] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:36.467193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.501093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc896ff8-342d-491e-a510-353e57269aca None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.510447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.536172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.536966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.537249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.537532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.548327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.549793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a79d96bb-7c07-477d-9eb0-8d31d9bbccf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:36.567119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:36.568164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:36.569786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:36.569786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:36.569786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:36.569786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:36.569786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 257/496] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:36.570204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df739e-cf4a-448d-b692-ee31c85708eb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.570396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fefd-92b6-4939-9cf8-403af9158353 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.570680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fefd-92b6-4939-9cf8-403af9158353 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.570981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fefd-92b6-4939-9cf8-403af9158353 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.572255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 258/497] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:36.573452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d72df2-f6fe-4ab2-8847-38b4566ba665 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.573612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d72df2-f6fe-4ab2-8847-38b4566ba665 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.573896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d72df2-f6fe-4ab2-8847-38b4566ba665 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.579055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 259/498] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:36.579055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d72df2-f6fe-4ab2-8847-38b4566ba665 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.611439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.612107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.612395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.612676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.613472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:36.613730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.615144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.617067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.624392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 240/499] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:36.625341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.679394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d72df2-f6fe-4ab2-8847-38b4566ba665 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.712768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.713410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.713694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.713972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.723483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba71c-fa84-4743-af4a-9cc17f4fe2d3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:36.741471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.757448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:36.758116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.758405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.758689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.759447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:36.759762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.761132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca0b9c-a8d7-4c9b-a801-6d9ebdb4fa51 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.773195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 241/500] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:36.774379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3f907-b995-44d4-92e3-e69d99264d05 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.774669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3f907-b995-44d4-92e3-e69d99264d05 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.774974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3f907-b995-44d4-92e3-e69d99264d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.787153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:36.788691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:36.790434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7be3f907-b995-44d4-92e3-e69d99264d05 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:36.791658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7be3f907-b995-44d4-92e3-e69d99264d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:36.841427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:36.859171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 242/501] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:36.860424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:36.860718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:36.861002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:36.877965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:36.879104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:36.898044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 260/502] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:36.899005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.958778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 243/503] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:36.958778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:36.967403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.012380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696cc3d-8d11-47d9-8d45-8e20c72b54ee None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.047898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.048695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.048981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.049260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.049809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a9701-f573-45ce-8092-3cfe0b21d603 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.059425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.060902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.066114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:37.079118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.080098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.080954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:37.080954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:37.080954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.080954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:37.080954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 261/504] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.081218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.089742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.090550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.090856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.091146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.098551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.100067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:37.115528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.116621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:37.117525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 244/505] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.118608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.181268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a43f9-38a0-44d6-87f4-441a3bf4d85e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.193552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.217347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.218131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.218670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.218987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.219386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d94d-5d83-4a0f-93d6-7f18de204a2a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.229165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:37.236700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:37.238480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 262/506] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.239640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.255520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.256305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.256683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.257043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.266788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.268255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:37.285708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.286763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.287658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:37.287658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:37.287658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.287658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:37.287658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 245/507] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.287658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.314324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.347079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0095f03-2674-4deb-bd20-35dc5049c407 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.379981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.380768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.381057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.381343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.384144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c32a28-798a-438c-b95f-469336c57ad8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.391677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.393153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:37.410420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.411383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.417374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:37.417374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:37.417374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.417374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:37.417374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 263/508] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.417374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.418872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.419547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.419837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.420119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.420845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:37.421158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.422533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.434806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 246/509] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:37.435686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.442796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.519691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac9e51-90cc-43bc-92d9-a9fcf13737dd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.521913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.533930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57595000-d599-4935-8059-ba6a1a50d509 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.554876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.555548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.555836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.556121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.556965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:37.557285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.558726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.567232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.567891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.568177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.568454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.569245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:37.569551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.570947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.578890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 247/510] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:37.579659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.595800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 264/511] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:37.596630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.676633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.677801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd456-4764-44a4-97c8-48f01ac0a90c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.695951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3d2ea-a69f-4bbc-9427-4ecff453d8f6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.711204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.711861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.712151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.712427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.713146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:37.713454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.714827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.728063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 248/512] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:37.729059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.729840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.730586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.730974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.731326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.732121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:37.732503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.733966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.743368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.745943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 265/513] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:37.746817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.832045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a47a-3559-4aec-a11c-395d8fa158ed None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.848650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d022a-82db-46d7-bcb7-7221cbfd8b79 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:37.867733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.868511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.868792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.869065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.875875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.877336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.882609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:37.883426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:37.883713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:37.883990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:37.891664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.892572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:37.893478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 249/514] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.894214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.899143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:37.900644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:37.902471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:37.919258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:37.920117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:37.920946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:37.920946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:37.920946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:37.920946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:37.920946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 266/515] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:37.921291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:37.970698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.003028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36876f71-2ebd-4a08-b595-f6034dd880a6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.023863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d80238-46dc-45f6-b930-ef68763f8c67 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.040928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.041572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.041957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.042237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.057711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.058499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.058783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.059099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.065882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.067379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:38.082000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:38.082830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:38.083692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 267/516] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:38.084148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.115552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.117361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.140985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.186476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.188509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41234e25-03f0-4def-9740-51a3ba40a5c6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.217997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 250/517] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:38.219119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.231454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.232176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.232530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.232879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.233658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.235205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.254680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 268/518] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:38 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:38.255509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.332216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38381766-8ab0-4b9f-afd7-625c6a5e6a91 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.371370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e28b0-b6d2-48ef-ae0a-d7c004f264c8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.383512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.385553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.386279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.386730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.387060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.388368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:38.388749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.390423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.409484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 251/519] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:38.410171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.417868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.419658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.419959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.420241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.420994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:38.421309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.422745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.440760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 269/520] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:38.440760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.449695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.539238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cff91-85fc-430e-bcd7-ca2decc0b03b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.561048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bda0e8-0256-49a4-b75a-5328b50f681e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.573738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.574406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.574693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.575008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.575802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:38.576114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.577553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.595551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 252/521] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:38.596334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.605516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.606320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.606605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.606909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.618326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.619887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:38.638250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:38.639172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:38.640160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:38.640160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:38.640160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:38.640160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:38.640160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 270/522] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:38.640441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.667482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.695390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.708848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e4ecd-a8e3-4c16-89c4-d332116badb7 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.749924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.750581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.750888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.751176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.752387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527926b-7c32-4435-ab53-c4d15c98d27e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.790543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.791308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.791594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.791871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.800345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.801851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:38.818195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:38.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:38.820737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 271/523] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:38.822768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:38.916056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:38.937142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680667e1-f46e-4dc2-813a-7e301f314c1a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:38.956016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.957494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.974258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:38.974943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:38.975259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:38.975540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:38.976374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=106467a9f0644177872555b7998599d1) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:38.976692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:38.978321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:38.984234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 272/524] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/role_assignments?user.id=106467a9f0644177872555b7998599d1&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:38.995888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.065772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.083177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 253/525] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:39.085092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.144694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095db2-1af3-4c0a-9b43-f9904706bd5b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.186499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.187213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.187506 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.187789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.188522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:39.188866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.190492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.199817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62ac28-9bd2-4782-985c-78a29c239b64 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.209909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 273/526] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:39.209909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.228374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.242900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.243510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.243799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.244082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.244923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:39.245235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.246691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.265752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 254/527] 10.222.0.41 () {56 vars in 1139 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:39.266590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.315682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.336389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776537-be8c-426a-b86e-9c00acfc9607 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.373950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e1580-34ea-45b4-b63f-163eb0618c1a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.374944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.376111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.376643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.377117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.389710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.391508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.410715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.411569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.411857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.412143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:39.412263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:39.413660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:39.414514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:39.414514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:39.414514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:39.414514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:39.416886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 274/528] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:39.418071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.427377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.429194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:39.445312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:39.446218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:39.447101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:39.451402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 255/529] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:39.452452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.489177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.535707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcc519-3b9c-4e4d-afe8-508021e5f324 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.568715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa906f-b083-4f4a-81e8-773aebbe3db8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.577572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.581443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.582108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.582475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.582740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.611209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.612314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.612835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.613307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.614220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:39.614826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.617530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.630911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 256/530] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:39.632209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.669763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.672110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.744663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b241-a907-4491-85d3-0f75cb41a9c4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.745732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.782702 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.783764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.784251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.784725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.785720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:39.786229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.788172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.792123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 275/531] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:39.793082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.802026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 257/532] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:39.804092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:39.834206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:39.919754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9e1f-7aa2-49f0-b2f6-abfb72fb4c13 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.924462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3a969-ff32-496f-ac17-c1112c1e6703 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:39.963337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.964366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:39.965154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.965260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:39.965433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.965716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.966108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:39.966735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:39.975968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.978420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:39.978695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.979901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:39.996243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:39.997110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:39.997940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:39.998779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:39.999585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:40.000970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:40.000970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:40.000970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:40.000970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:40.000970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 258/533] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:40.001599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.003972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 276/534] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:40.006606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.031274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:40.107916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:40.136034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fcd79-7dcf-42af-b747-03a6dc67e467 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:40.156431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de39158-40d5-4bf8-910a-edb78c8422e1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:40.185111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:40.186188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.186725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.187244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.188331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=d0fa7ced305e4d2fa0d791675f839453) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:40.188868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:40.190690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:40.197901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 259/535] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/role_assignments?user.id=d0fa7ced305e4d2fa0d791675f839453&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:40.199285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.201206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:40.201881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.202164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.202438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.203397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:40.203504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:40.205509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:40.219770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 277/536] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:40.232067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.325578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78f2e-ca50-4b21-ab58-f13da90db3c4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:40.367141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:40.368181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.368668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.369185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.370174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:40.370792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:40.372466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:40.374905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:40.389200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 260/537] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:40.395071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.400361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:40.441182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c90e3-3edb-4015-a439-b4be943138e8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:40.518188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa920d3c-2ab3-4756-b25a-ec5c46d9c4ff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:40.529491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:40.530625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.531130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.531606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.548024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:40.549527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:40.569596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:40.570585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.571119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.571599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:40.572240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:40.573067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:40.573891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:40.575167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=b0208c5889a74870b764c7eb954242bb) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:40.575704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:40.577336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91592088-09ed-4684-9106-a6fc66b2db8a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:40.583268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 278/538] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:40.584141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 261/539] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/role_assignments?user.id=b0208c5889a74870b764c7eb954242bb&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:40.589874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.763312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbdf1-8ed2-438a-a243-c3ac07531b84 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:40.808993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:40.809965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.809965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.810206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.811031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:40.811344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:40.813148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc9ad5-2e7b-4138-b459-20bdf2ac617e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:40.825589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:40.832984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d947fd-3c39-4670-950b-011934584ad5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.833336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d947fd-3c39-4670-950b-011934584ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.833716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d947fd-3c39-4670-950b-011934584ad5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.836586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 262/540] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:50:40.839236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 279/541] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:40.844911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d947fd-3c39-4670-950b-011934584ad5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:40.849385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:40.850481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:40.850968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:40.875991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:40.877483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:40.970668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:41.018948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 280/542] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:41.030378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d947fd-3c39-4670-950b-011934584ad5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:41.061245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:41.082701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:41.083411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.083698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.083974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.085610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=2665b33ffa38466a98a5e4ea838916c5) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:41.086119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:41.088670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:41.102799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 263/543] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:40 2022] GET /identity/v3/role_assignments?user.id=2665b33ffa38466a98a5e4ea838916c5&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:41.223609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c2a55-122b-41c6-945b-b9aff4e32746 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:41.271308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:41.278654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b9300-3edd-406f-9294-84b066b519f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:41.279514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b9300-3edd-406f-9294-84b066b519f9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.279882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b9300-3edd-406f-9294-84b066b519f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.280345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b9300-3edd-406f-9294-84b066b519f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.297041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b9300-3edd-406f-9294-84b066b519f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-321b9300-3edd-406f-9294-84b066b519f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:41.305799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:41.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:41.307570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:41.311529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 281/544] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:41.317152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:41.467576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9817a-0c01-4d43-ba90-272eef28273c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:41.490585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ee0cf-c9da-4e68-84b1-3bd8d4be40c8 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.491022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ee0cf-c9da-4e68-84b1-3bd8d4be40c8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.491207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ee0cf-c9da-4e68-84b1-3bd8d4be40c8 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.492692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 282/545] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:41.498062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:41.499609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.500360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.500919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.516384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:41.517450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.517950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.518420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.519415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:41.519920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:41.521759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:41.529180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:41.531546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:41.541109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 264/546] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:41.546035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:41.673093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 283/547] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:41.707672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:41.733512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62566b-4e09-4df7-9b16-14ebaee12b21 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:41.785382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:41.786097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.786387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.786669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.798158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:41.799665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:41.817631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:41.818616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:41.819843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:41.819843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:41.819843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:41.819843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:41.819843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 265/548] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:41.828228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:41.867277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605f112-afa2-4742-a922-ebcb3bac867c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:41.901657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:41.927202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:41.932511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:41.933510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:41.933969 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:41.935177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:41.956185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:41.967002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:41.968833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:41.969901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:41.983862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 284/549] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:41.992653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.041649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9969c-7776-41ab-adc2-28d350a9e85f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.136446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.137128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.137423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.137708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.138450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:42.138763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.139734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0652ca8-ea6e-4866-9be9-7ccd6f571d62 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.140171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.169388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 266/550] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:42.175063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.181374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.182225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.182656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.183103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.184786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:42.189397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.191042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.213909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 285/551] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:42.216327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.302711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:42.364674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a33a6-46a1-4ed1-8ee7-ee1c3b63627a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.383277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:42.407984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.409051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.410956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.411415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.427130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.429764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.446528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edb44e-acb6-4825-a9ea-e14d676c58c1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:42.470222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:42.471593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:42.472881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 267/552] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:42.479814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.496291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.497364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.497830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.498263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.523090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.525566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:42.554927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:42.556015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:42.557159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:42.557159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:42.557159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:42.557159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:42.573850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 286/553] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:42.577330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.683810 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:42.694093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506116fd-bbd4-4dc0-b84c-60cb88547a5f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.719588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41dddd-c2dc-41b7-8a5d-7bb448e7bf7a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.729751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.730734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.731258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.731741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.732700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:42.733213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.734885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.752657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 287/554] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:42.754190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.827638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.828880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.828880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.828880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.829644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:42.829955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.831462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.863605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1305a66-29b7-40b8-974d-33b22d530aa4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:42.885261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 268/555] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:42.895872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.902731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:42.904162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:42.904651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:42.905132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:42.912869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:42.914574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:42.930901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:42.931948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:50:42.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:42.941874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 288/556] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:42.945722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:42.970868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:42.971194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.078007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f734a-e101-45ce-8df7-51a09e6718e2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.088835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ea1d4-f8d6-43d2-96dd-8729261702ba None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.124533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6201fa9-33a0-4a45-80a5-955d9ed002b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.125222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6201fa9-33a0-4a45-80a5-955d9ed002b5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.125574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6201fa9-33a0-4a45-80a5-955d9ed002b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.125863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6201fa9-33a0-4a45-80a5-955d9ed002b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.127804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.128815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.129344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.129896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.131546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:43.132100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.133888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f970-7fd6-4338-bdd4-475e730e3681 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.156737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 289/557] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:43.163329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db2784-7d21-45cf-8cf8-73507fd018b9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.163877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db2784-7d21-45cf-8cf8-73507fd018b9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.164381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db2784-7d21-45cf-8cf8-73507fd018b9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.166650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 290/558] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:43.167732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db2784-7d21-45cf-8cf8-73507fd018b9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.218386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6201fa9-33a0-4a45-80a5-955d9ed002b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.219947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6201fa9-33a0-4a45-80a5-955d9ed002b5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.247849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.248842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.285030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db2784-7d21-45cf-8cf8-73507fd018b9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.320642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.321453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.321747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.322030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.334486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.340094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 269/559] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 11:50:43.340287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.340369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.340369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.340369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.358621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:43.359851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e9989ebc-c0b3-4dad-853f-215501c85cbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:43.360057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:43.360940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:43.361825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:43.361825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:43.361825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:43.361825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:43.361825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 291/560] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:43.362889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ce826-b917-469d-a78b-d46e3dbc1417 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.363163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ce826-b917-469d-a78b-d46e3dbc1417 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.363445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ce826-b917-469d-a78b-d46e3dbc1417 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.364685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 292/561] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:43.365752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ce826-b917-469d-a78b-d46e3dbc1417 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.446864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 270/562] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:43.446864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.482712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ce826-b917-469d-a78b-d46e3dbc1417 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.507188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.507836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.521980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.523089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.523381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.523657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.535698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.538139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-683a172a-b845-4371-8a03-df3dd3e24271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:43.556689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:43.557628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:43.558513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:43.558513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:43.558513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:43.558513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:43.558513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 293/563] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:43.559117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.559409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.559692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.566033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff78d2-ae18-4ea6-935b-f59b5780856a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.576508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:43.577709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:43.604853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.605884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.606371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.606830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.607960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:43.608497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.610039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.623158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 271/564] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:43.624205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.650293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 294/565] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:43.651512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.737165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bdc81a-c183-49cc-b9b7-f681028988fb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.750012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.752442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d05f6-b66f-4e68-8035-80b83729dce5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.755025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:43.771222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.772018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.772308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.772585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.782434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:43.786469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.787696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.787981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.788256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.788972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:43.789282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.790673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b4b49-0d18-4ca6-919d-8aead4842c06 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:43.790853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:43.791947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:43.792828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 272/566] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:43.793533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.807559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 295/567] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:43.808844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa0a39-9a45-440b-bb9c-690d1cc9e9ae None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.809138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa0a39-9a45-440b-bb9c-690d1cc9e9ae None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.809419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa0a39-9a45-440b-bb9c-690d1cc9e9ae None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.810996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 296/568] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:43.811368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa0a39-9a45-440b-bb9c-690d1cc9e9ae None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.903055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd20a3-91d2-4d2f-9217-4114ffb1bec2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.917575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa0a39-9a45-440b-bb9c-690d1cc9e9ae None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:43.936671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.937451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.937738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.938016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.944996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:43.946464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:43.951180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:43.951970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:43.952251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:43.952529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:43.961229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:43.962236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 273/569] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:43.963169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:43.966332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:43.974253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:43.975714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 297/570] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:43.976665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.000464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.006061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.069636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2697cc-aff5-4e72-b370-3061875e7ce5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.082722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0c172-0024-4d29-a469-f862738e224c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.104234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.104903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.105197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.105476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.106220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:44.106531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.107942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40a458-ed41-41b4-b7a5-0f6ff8d3c48a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.115830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 274/571] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:44.117082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.117383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.117481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.117673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.118263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.118545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.118821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.125762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.127269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.133629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:44.134823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:44.141104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:44.142074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:44.142981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 298/572] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:44.143468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.202638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 275/573] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:44.203737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.229540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.236660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.252989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab790-74dc-4728-b411-b0280ab7c77d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.287617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.288272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.288558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.288839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.289622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:44.289932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.291430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.303823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 299/574] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:44.304612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.310482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bc2e8-f5ad-49cc-82aa-76f36698ba52 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.344291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.344953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.345238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.345515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.346263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:44.346573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.347978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.356681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 276/575] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:44.357487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.404584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6de45e-b10d-45c7-97a6-6fa66419a246 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.438318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.439139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.439432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.439716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.443534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.451049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.452454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.455583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.458310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5921-5a66-40bd-887e-9c2d755becac None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:44.471682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:44.472743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:44.473676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:44.473676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:44.473676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:44.473676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:44.473676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 300/576] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:44.473954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.495768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.496431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.496718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.497000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.497780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:44.498095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.499579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.511867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 277/577] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:44.512634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.576468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974a64f-5ea2-436e-9a80-28a58481ebe2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.609959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.610752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.610975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98f26b-72c6-4fd9-a07a-f80e2ae3bdfc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.611128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.611338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.623502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:44.631736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:44.632784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:44.633621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 301/578] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:44.634636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.648806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.649727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.650007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.650281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/106467a9f0644177872555b7998599d1/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.664803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=106467a9f0644177872555b7998599d1, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.666251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.669319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.680119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 278/579] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:44 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/106467a9f0644177872555b7998599d1/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:44.681030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.691191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.753849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e6b9c-35a1-4bec-9e97-809e4cb664f8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.792031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.792836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.793125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.793405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.803083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da71a6-303a-41de-ad06-b5eb2a0ee869 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.807199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.808709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:44.827928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:44.829086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:44.829985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:44.829985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:44.829985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:44.829985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:44.829985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 302/580] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:44.830269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.846598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.847306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.847591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.847872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.848598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:44.848907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.850577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:44.867947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 279/581] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:44.867947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:44.931866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.952495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e94ac-5dca-418e-a0bb-a18ca76aa663 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.963575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:44.978959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de54b5-9bde-4a46-8065-34c1be5514a2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:44.991588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:44.992246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:44.992532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:44.992876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:44.993632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=0a914be52bf745d4b4647f482460e734) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:44.994185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:44.995446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.017761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 303/582] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/role_assignments?user.id=0a914be52bf745d4b4647f482460e734&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:45.019926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.025195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.026563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.026563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.026563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.027382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:45.027699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.029192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.044588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 280/583] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:45.045652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.160507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cedd0-6ded-4907-b9a9-dd336806f127 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.162027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fe20d-4c17-4f50-a715-6d0decb2985a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.181813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:45.202815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.203691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.203983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.204302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.206944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.207837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.208325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.208878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.210114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:45.210529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.211959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.214829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.216331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.226100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 304/584] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:45.227318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:45.231629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:45.232581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:45.233715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 281/585] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:45.234863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.271088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:45.356933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652634b3-9537-4ccd-97cf-1bfeb25871c0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.368677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45073f6-4e54-49c6-969d-6e123d8fb767 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.393849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.395170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.395691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.396175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.405577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.407399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.411203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.412039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.412325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.412609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.423974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:45.424402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:45.425315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:45.426364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:45.426933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.432941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 282/586] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:45.432941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:45.455995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:45.456939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:45.458258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:45.458258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:45.458258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:45.458258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:45.462581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 305/587] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:45.463498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.482120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:45.552088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67501c6-5002-4d1c-92a6-3af8b783709f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.570256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:45.591640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93339fda-27dd-492a-bbba-6a9f505cf1cf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.600630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.601590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.602551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.603091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.604572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:45.605332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.607323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.631502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 283/588] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:45.632278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.632897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.633006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.633434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.633782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.634690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:45.635141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.637189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.665299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 306/589] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:45.665299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.760683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:45.770641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3f08-984c-4a33-9cf7-9ed638778d46 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.792697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76993b-729f-48d4-8642-32fd3e973c33 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.808448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.809126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.809778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.809778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.810556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:45.810956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.812414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.829512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 284/590] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:45.830566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.835346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.836578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.837103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.837564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.851669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.853660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:45.855776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:45.875751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:45.876710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:45.877600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:45.877600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:45.877600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:45.877600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:45.879761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 307/591] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:45.880824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:45.942624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37ed8d-4be1-4314-a63b-46ce9a56319d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:45.978232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:45.979233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:45.979935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:45.979935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:45.992139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:45.993766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:46.014300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:46.015295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:46.016186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:46.016186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:46.016186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:46.016186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:46.018931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccca9f2-4b60-4265-813d-d450d28b532d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.020408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 285/592] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:46.021372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.033410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.062478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.063695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.064204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.064663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.073226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.075069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:46.091302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:46.092275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 11:50:46.093469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:46.096396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 308/593] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:46.097478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.132228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557296a6-5bce-4258-ad7a-82462ba52159 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.161270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.169759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.170518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.170927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.171287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.172189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:46.172570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.174123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.187674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 286/594] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:46.188673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.216412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c10bb-b0d4-4519-9006-aa4e45bd0e98 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.260028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.261196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.262455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.262979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.263479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.264323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:46.264740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.267038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.282951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 309/595] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:46.284208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.305622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a1462-a848-488e-a266-1bb5f2fec974 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.343677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.344276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.344672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.344957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.345804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:46.346119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.347692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.373274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 287/596] 10.222.0.41 () {56 vars in 1139 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:46.374260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.399761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54caf6-e67d-4edf-a809-d903f5fafe17 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.415300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.437330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.438170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.438418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.438734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.446306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.447914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:46.463605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:46.464791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:46.465889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 310/597] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:46.466404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.488481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b976-b14b-488f-b6a6-f213c46b7127 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.516511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.526824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.528020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.528467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.528909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.537433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.539199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:46.554045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:46.555329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:46.556444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:46.558366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 288/598] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:46.559332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.585886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758bbc9-27fb-4566-b711-259738a50929 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.632938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.633667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.633956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.634236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.645956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.647733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:46.666997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:46.668140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:46.669179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:46.669179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:46.669179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:46.669179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:46.669179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 311/599] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:46.669179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.686526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148027c9-a938-4e4d-b964-c1107dc189f2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.701128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.730141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.731154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.731590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.732038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.733090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:46.733566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.735436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.749855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 289/600] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:46.751110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.783915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8de3b-72bd-4877-8300-88170241dde7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.803008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.822700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.823429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.823720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.824002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.824795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:46.825110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.826582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.844939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 312/601] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:46.845807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.863296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb463cf-4fb1-4dbb-b5b7-842437be0ca4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.902338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.903521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.904078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.904613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:46.905537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:46.906055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:46.907895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:46.922352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 290/602] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:46.923619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:46.936456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:46.952404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef93f24-e5f9-4b61-8bd5-ee3d1c38a14f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:46.988938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:46.989892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:46.990386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:46.990679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/d0fa7ced305e4d2fa0d791675f839453/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.016356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=d0fa7ced305e4d2fa0d791675f839453, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:47.018224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:47.036298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 313/603] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:46 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/d0fa7ced305e4d2fa0d791675f839453/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:47.037957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:47.047976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71878b-e368-4596-99c8-7554690242d0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.071281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:47.089055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458a120-4181-4eab-aeb0-374395c8ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:47.090396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458a120-4181-4eab-aeb0-374395c8ef9e None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.091025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458a120-4181-4eab-aeb0-374395c8ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.091566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458a120-4181-4eab-aeb0-374395c8ef9e None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/b0208c5889a74870b764c7eb954242bb/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.114023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458a120-4181-4eab-aeb0-374395c8ef9e None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=b0208c5889a74870b764c7eb954242bb, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:47.116525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458a120-4181-4eab-aeb0-374395c8ef9e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:47.147104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 291/604] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:46 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/b0208c5889a74870b764c7eb954242bb/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:47.151323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595dba-61e6-4121-ba01-bfa987db4378 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.152690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595dba-61e6-4121-ba01-bfa987db4378 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.153303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595dba-61e6-4121-ba01-bfa987db4378 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.157379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 292/605] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:47.164442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.165045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.165642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.196697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:47.198817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:47.218235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64c5c-4a7c-4c8e-85be-d92c6ece732b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.289833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 293/606] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:47.313090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:47.313923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.314348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.314899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.323026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:47.334857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:47.335608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:47.336661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:47.359859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:47.360827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:47.361743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 314/607] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:47.368101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:47.458780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:47.475565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425876e9-46e8-442b-b185-8a7fd08ad4db None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.485154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544c847-d038-4ca0-81b7-e420e9dc65aa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.513614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:47.514585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.515051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.515460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.520511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:47.521767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.522135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.522525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.523341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:47.523655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:47.525116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3601c-6de4-4da1-8e3e-61862ae754f5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:47.527716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:47.538574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 315/608] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:47.540057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:47.541114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:47.541968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:47.544111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56d1e3-a417-40d5-b1fc-28f192a0197e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.544411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56d1e3-a417-40d5-b1fc-28f192a0197e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.544691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56d1e3-a417-40d5-b1fc-28f192a0197e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.545415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 294/609] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:47.550017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 316/610] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:47.551195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:47.551397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56d1e3-a417-40d5-b1fc-28f192a0197e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:47.616992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:47.674195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56d1e3-a417-40d5-b1fc-28f192a0197e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.681395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d1d99-c5f5-4dce-870c-019e0268209e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.712303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:47.713052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.713440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.713825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.714699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:47.715150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:47.716973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ea8e9-153c-4b08-8709-3fe4ec3e5821 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:47.728190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:47.729303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.729738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.730161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/2665b33ffa38466a98a5e4ea838916c5/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.731384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 317/611] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:47.732863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:47.733317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:47.733699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:47.748390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=2665b33ffa38466a98a5e4ea838916c5, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:47.750621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:47.755243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:47.757933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:47.767299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 295/612] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:47 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/2665b33ffa38466a98a5e4ea838916c5/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:47.772837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:47.775251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:47.894656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 318/613] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:47.969582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d577f3-5615-4d79-8b2d-b47c427ecc52 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:47.975144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.046123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.047068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.047355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.047633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.060143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:48.063393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.066045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:48.090459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:48.091442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:48.093712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:48.093712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:48.093712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:48.093712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:48.093712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 296/614] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:48.098152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.124678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f379560-5dd4-4c11-b74d-a3d362650efb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.149078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:48.170944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.171813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.172109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.172394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.189893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:48.204767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:48.205707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:48.206751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 319/615] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:48.213550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.245249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8ebb5-2b75-4dab-b29f-2aef8624f7de None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.326302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.328167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.328837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.330498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.332247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:48.336188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.337843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.359614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 297/616] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:48.362541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.427857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43ff3b-9801-4935-a057-1616c8f32fa1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.445540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:48.512088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4478-489b-44fa-b588-ccf3196caf1c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.527353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:48.527654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.528402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.528698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.529000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.529746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:48.530060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.531514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.543402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 320/617] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:48.550866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.573174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.574349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.574880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.575483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.594865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.596503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:48.627291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:48.628301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:48.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 298/618] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:48.635785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.659888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20824c-f4d8-4ce4-929b-0b16a9ec1876 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.695857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.696473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.696874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.697156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.708158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.710171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:48.727939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:48.728986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:48.730488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:48.730488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:48.730488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:48.730488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:48.730488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 321/619] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:48.734934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.762573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:48.785050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d01cd9-c6b9-487a-b2be-ba2d54937ddf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.821768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.822857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.823357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.823862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.824959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:48.825495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.827287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.846655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 299/620] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:48.857453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:48.873190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e988634-71f0-4a1f-96dc-788d8409da99 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:48.910865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:48.910865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:48.910865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:48.910865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:48.911291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:48.911291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:48.911291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:48.915187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:48.924143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 322/621] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:48.929655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.021575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:49.033687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2664850-d99a-4c82-bd6b-0a77a73dbda9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.047020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ad6b7-ce02-400c-b3b9-dce8eef4615c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.074664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.075493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.075778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.076054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.085525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.087017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:49.102160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:49.103191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:49.104139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 323/622] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:49.109192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.124458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.125471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.125958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.126457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.231313 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca014-5fd2-4b72-a678-bcd22e77cf59 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.278889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.279506 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.279794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.280077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.280925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:49.281241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.282697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.287269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.288843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.319216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:49.325972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 324/623] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 11:50:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:49.326974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.355208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:49.506439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 300/624] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:49.510766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec13af-c187-4f15-ad0a-2bc1555a96c4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.511718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.580877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.581673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.581955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.582231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.600462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.601937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:49.620131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:49.621068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:49.621963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:49.621963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:49.621963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:49.621963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:49.621963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 325/625] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:49.627759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.686147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:49.728798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50f9b-c319-46f8-9489-c6f8b7f09a72 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.739452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db352d-fcb0-4d2b-9638-a85575f32130 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.764054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:49.774634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.775746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.776237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.776860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.777993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.778717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.779119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.779474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.780255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:49.780629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.782083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead8d5-26ab-401e-8266-76b62948889e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.789712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.791428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.794906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 326/626] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:49.796220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f82bb-1332-461d-bd95-33c1c51e6b11 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.796510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f82bb-1332-461d-bd95-33c1c51e6b11 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.796790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f82bb-1332-461d-bd95-33c1c51e6b11 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.802074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 327/627] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:49.802074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f82bb-1332-461d-bd95-33c1c51e6b11 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-52da80ac-59a6-405f-869e-f147edc9d241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:49.817024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:49.818916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:49.820181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:49.820181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:49.820181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:49.820181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:49.821680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 301/628] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:49.823055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.823715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.824088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.846817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:49.848057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:49.926320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f82bb-1332-461d-bd95-33c1c51e6b11 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:49.934686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 302/629] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:49.935565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:49.946002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:49.970230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:49.971072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.971362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:49.971643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:49.979640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:49.981110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6e83ac2f-d055-4529-8a8c-a6d81f8cad80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:49.996856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:49.997818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:49.998720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 328/630] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:49.999712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e418876-5c71-43e1-a7f3-18b98b104e58 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:49.999885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e418876-5c71-43e1-a7f3-18b98b104e58 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.000156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e418876-5c71-43e1-a7f3-18b98b104e58 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.001521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 329/631] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:50.002071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e418876-5c71-43e1-a7f3-18b98b104e58 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.063588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764536e-d9e5-4311-9964-b8e9a9c1490d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.074070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.106350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.107041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.107380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.107733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.108529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:50.108957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.110387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad061f-6af2-4cb3-95f2-7242ec35e1e8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.113216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e418876-5c71-43e1-a7f3-18b98b104e58 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.128668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 303/632] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:50.128668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.128668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.128668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.146634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:50.147948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:50.153857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.154651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.154962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.155240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.165994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:50.173698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:50.174605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:50.175475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:50.180359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 330/633] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:50.181157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.214713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.223525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 304/634] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:50.224938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.290885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c029f3-1e84-4410-8d02-110ab90cb50f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.326816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.327610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.327904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.328186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.328974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:50.329289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.331026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.336488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.340738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518edbb-f320-4250-9acf-2c1081c98a9e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.354995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 331/635] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:50.355973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.382554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.383306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.383588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.383859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.391131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.392577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:50.406921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:50.407898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:50.408789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 305/636] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:50.409274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.460715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.469952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f174afa-6447-4412-a2fc-997e6dd64b41 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.503556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.504219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.504502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.504778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.506204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:50.506620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.508027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.511323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5882b7-7aa4-49e3-b4ca-fb5fce00dc7a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.515956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 332/637] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:50.516552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.549562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.550347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.551286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.551286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.560770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:50.568857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:50.569855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:50.570696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 306/638] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:50.571418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.589899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.619638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b0981-27bf-4399-9e1f-21490cfe9a0c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.652790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53645-ec1e-4c7f-a1b6-ee154d68e840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.653612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53645-ec1e-4c7f-a1b6-ee154d68e840 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.653902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53645-ec1e-4c7f-a1b6-ee154d68e840 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.654182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53645-ec1e-4c7f-a1b6-ee154d68e840 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/0a914be52bf745d4b4647f482460e734/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.668418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53645-ec1e-4c7f-a1b6-ee154d68e840 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=0a914be52bf745d4b4647f482460e734, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.669909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53645-ec1e-4c7f-a1b6-ee154d68e840 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.670818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e56b-6fc3-42cd-b6d8-0d75e1aed525 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.682942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 333/639] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:50:50 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/0a914be52bf745d4b4647f482460e734/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:50:50.684279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2736a4-6ff2-47ee-8ab5-5a6794031730 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.684578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2736a4-6ff2-47ee-8ab5-5a6794031730 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.684859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2736a4-6ff2-47ee-8ab5-5a6794031730 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.686148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 334/640] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:50.686905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2736a4-6ff2-47ee-8ab5-5a6794031730 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.691649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.729259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.729943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.730232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.730512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.731350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:50.731663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.733131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.747687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 307/641] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:50.748315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.815567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2736a4-6ff2-47ee-8ab5-5a6794031730 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.838108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:50.856939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.857739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.858021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.858296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.863522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900006b1-fc67-4b87-800b-f49cec0fd8d2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:50.870259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.871762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-bcb9c002-a96f-4b98-a7ff-1d60a5b9dc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:50.891364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:50.892545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:50.893485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:50.893485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:50.893485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:50.893485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:50.893485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 335/642] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:50.894080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.894414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.894685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.910012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:50.910633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:50.910956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:50.911264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:50.912105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:50.912418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:50.912997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:50.913854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:50.914388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:50.922387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 308/643] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:50.923228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:50.974056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.002862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 336/644] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:51.002862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.050443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d5e1-1d42-4e3c-88d6-54e523b2a0b1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.086644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.087333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.087622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.087903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.088737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=106467a9f0644177872555b7998599d1) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:51.089057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.090507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.097272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 309/645] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/role_assignments?user.id=106467a9f0644177872555b7998599d1&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:51.098204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.108556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.132858 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f3632-e2e1-429d-a362-06bc8c9e18ad None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.179509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.180551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.181006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.181458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.182420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:51.183054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.184891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.200042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 337/646] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:51.201302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.240194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f8db9-a742-4a0a-a6e3-26e0182fce90 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.258217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.290447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.291315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.291600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.291880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.307855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.309333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.326930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6055-cb6a-441c-82c3-f8d726b9521e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:51.331594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:51.332480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:51.333617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:51.333617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:51.333617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:51.333617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:51.333617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 310/647] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:51.334367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.373550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.374360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.374659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.374996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.387906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:51.396643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:51.397620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:51.398460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:51.402652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 338/648] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:51.403544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.403667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.467066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0b8fe-9a21-4bea-a41e-b486452c84ea None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.515562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.517273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.517872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.518450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.524654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793c749-88c3-473c-9cd9-2906606af416 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.528906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.530583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.532863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:51.550724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:51.551780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:51.553287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:51.556559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 311/649] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:51.557648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.568055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.568942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.569333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.569719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.570549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:51.571065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.572895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.586366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 339/650] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:51.587479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.673445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a543205-1360-4640-a122-e3b7284654e8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.691456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.714444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.715225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.715516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.715803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.716629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:51.716921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.718633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.725632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bbb7-d2aa-4eb7-84dd-ab98ba4ff09a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.732660 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 312/651] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:51.733631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.770573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.773123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.773591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.774054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.775097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:51.775587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.777762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.805771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 340/652] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:51.805771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.844229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.850866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7d8bc-26c1-47da-97f9-6824ec9174f5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.898661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.900153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.901475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.901975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.912866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.915419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:51.932555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb986e1-b4bc-4e57-886c-a9b3f606c8fa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:51.936181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:51.937097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 11:50:51.938042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:51.941294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 313/653] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:51.943404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:51.976369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:51.977187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:51.977470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:51.977747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:51.989305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:51.990207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:51.992282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:52.013863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:52.014804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:52.015715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:52.015715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:52.015715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:52.015715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:52.015715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 341/654] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:52.016461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.060466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf7a1-58c7-4131-a956-0fa4761b8e42 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.096883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.097861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.098345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.099371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.102956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.128509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef5a2-3a35-4cab-9115-994ae95db612 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.166882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.167611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.167841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.168124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.169061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:52.169377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.170819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.176054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.177973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.194797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 342/655] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:52.195692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.219248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.289290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 314/656] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:52.290667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.307160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da11ab-c991-43e4-8d5e-7a339154b83f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.346615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.347372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.347655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.347928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.348667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:52.349586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.351689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.353461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.366515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 343/657] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:52.367567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.404408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04e8c-13fd-4e98-b623-5ac258e3d2a9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.442067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.443199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.443685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.444160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.453053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.481251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61b309-9fe9-4408-ba34-08c67e6f9a26 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.518712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.519097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.519969 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.520396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.521393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.521548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.533633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.535387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:52.555488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:52.556523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:52.557388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:52.557388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:52.557388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:52.557388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:52.559646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 344/658] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:52.560605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.634406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.663085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 315/659] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:52.664316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.711363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc78ea1-a888-415b-87c3-64d1f7714f87 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.715489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.763611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.764876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.765165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.765448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.775953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.777976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.790819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bb7dd-598d-4650-875a-dbb8b4f38d5b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:52.797104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:52.798019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:52.799394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 345/660] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:52.800035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.831419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.832498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.832987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.833712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.835214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:52.836110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.838951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:52.856803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 316/661] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:52.857965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:52.929622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d006b3-0263-4831-88b6-290091b408b0 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.937873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.976254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:52.977456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:52.977747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:52.978031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:52.980189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:52.984477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d243f-5960-4ea8-8339-a6640c38f9fb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:52.992962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:52.994933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:53.016570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:53.018471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:53.019501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:53.019501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:53.019501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:53.019501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:53.023156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 346/662] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:53.024121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.035015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.035746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.036035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.036445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.037767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:53.038083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.039720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.069130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 317/663] 10.222.0.41 () {56 vars in 1139 bytes} [Mon Jun 6 11:50:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:53.069130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.139837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d97d12-53c0-4598-b0e1-c7bf7ddead08 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.175124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.175959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.176245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.176530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.180083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b58a08-035b-4132-8e9a-36162e1c50cb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.184972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.186612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3ac29f42-1ebc-41a8-980e-17c9e824fa3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:53.203759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:53.204742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:53.205627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 347/664] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:53.206429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9babc0-98c3-477c-9404-e6bc5de81fe9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.206610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9babc0-98c3-477c-9404-e6bc5de81fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.206923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9babc0-98c3-477c-9404-e6bc5de81fe9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.210535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 348/665] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:53.210535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9babc0-98c3-477c-9404-e6bc5de81fe9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.228275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.229056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.229581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:53.230088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.230379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.231200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:53.231469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.232810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.247156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 318/666] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:53.247995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.270507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:53.319327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9babc0-98c3-477c-9404-e6bc5de81fe9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.358347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.359034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.359325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.359795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.362433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f6669-0268-47c0-93aa-2a2c3dca8b49 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.402221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.402803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.403133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.403420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.404216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:53.404533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.405969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5d5b-fd4b-4dcf-ac68-d55a87e9cbf7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.418582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 319/667] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:53.419853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.420151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.420435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.437001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:53.438187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:53.448730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.450520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.469876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:53.514408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 320/668] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:53.515297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.524465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:53.565873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 349/669] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:53.566710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.620521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fc54e-92b6-4083-ad07-b136a0c841c3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.654973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.655695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.655981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.656257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.663371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.664743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00fe5-fb1c-48a0-ad32-f5704d8afbf6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.664846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c194353f-4634-436f-a1aa-9ac9895d6ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:53.681528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:53.682505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:53.683723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:53.684161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:53.684233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 321/670] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:53.684233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91149ef0-74eb-40ec-a07f-c59e991ab428 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.684363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91149ef0-74eb-40ec-a07f-c59e991ab428 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.684709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91149ef0-74eb-40ec-a07f-c59e991ab428 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.686005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 322/671] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:53.686744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91149ef0-74eb-40ec-a07f-c59e991ab428 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.705959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.706616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.706930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.707214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.708716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=b0208c5889a74870b764c7eb954242bb) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:53.709035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.710595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.716521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 350/672] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/role_assignments?user.id=b0208c5889a74870b764c7eb954242bb&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:53.717543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.790491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:53.830519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91149ef0-74eb-40ec-a07f-c59e991ab428 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.832558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d1c5d-b7c8-47e7-b434-e9c4a3c6e26b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:53.868991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.870193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.870670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.871170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.874071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:53.874887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.875185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.875472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.885004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:53.887440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:53.888952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:53.894149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:53.895288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:50:53.896960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:53.898690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 323/673] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:53.899517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-37740276-caba-4fa2-bef1-192cc23cc5c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:53.908670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:53.909566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:53.910480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:53.910480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:53.910480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:53.910480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:53.911919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 351/674] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:53.913463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:53.913758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:53.914045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:53.931199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:53.932405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:53.961963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.017896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 352/675] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:54.019028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.031576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255299-60b2-46cc-bdde-d6c8c5e4093e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.072614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.084930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.085603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.085923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.086186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.087007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.087324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.088769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.101969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 324/676] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.103017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.137340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b82905-5de9-41ec-ada5-556383f7eaaa None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.173357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.173950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.174240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.174526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.175310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.175844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.177098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.185991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 353/677] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.187770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.214095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1d232-251c-4f81-96f8-22194ce7c894 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.225889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.249846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.250512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.250800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.251107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.251832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.252140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.253510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.265796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 325/678] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.266612 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.312672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8d6c7-99e9-4ca1-96e1-e8d18df3d394 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.320179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.356296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.357304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.357593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.357873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.358959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.360407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:54.375943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:54.377120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:54.377967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 354/679] 10.222.0.41 () {56 vars in 1129 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:54.378831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.396179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd5661-1d1b-498e-afd0-c1cf80aa9649 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.431025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.431870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.432156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.432601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.433720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=d0fa7ced305e4d2fa0d791675f839453) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.434040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.435502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.442000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 326/680] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/role_assignments?user.id=d0fa7ced305e4d2fa0d791675f839453&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.444701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.487920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.500848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0493d-9248-44a7-8b37-cdb66b03324b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.548272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.549462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.549816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.550181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.566290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.569405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.575483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3b79c-6d16-4a37-89d4-df98c5b1967b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.589212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:54.598916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:54.600104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:54.601006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:54.601006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:54.601006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:54.601006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:54.605500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 355/681] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:54.606487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.622373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.623690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.624192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.624679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.633783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.635774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:54.653163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:54.654188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:54.655117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:54.656885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 327/682] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:54.661688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.731084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358e438-7721-4332-bbe2-bc76d733ddf5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.776677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.777230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.777466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.778287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.779490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.780215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.782218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.783732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1123-e3f4-44ae-a86d-4ccc829bb72a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.787530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 356/683] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.788343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.802898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.820388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.821050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.821332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.821612 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.822329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.822624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.824055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.835882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 328/684] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.836715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.910172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af714c8-9432-4947-bd4a-ada77115d725 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.931233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:54.939803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8a460-9030-4016-b503-777052d9a8ab None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:54.950049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.951100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.951619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.952145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.953134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.953643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.955361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.969787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 357/685] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.971042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:54.978344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:54.979044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:54.979328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:54.979605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:54.980318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:54.980622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:54.982522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:54.988916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 329/686] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:54.990617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:55.055386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:55.095553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdc83-f0a0-4d08-9bfd-6d793335d636 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:55.134984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:55.136220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:55.136725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:55.137210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:55.146063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:55.148405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:55.167647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:55.168657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:50:55.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:55.172510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 358/687] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:55.178414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:55.233848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf35e-4abc-4e50-9749-b56895a66e18 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:55.240544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:55.286325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:55.287119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:55.287413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:55.287698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:55.288829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=2665b33ffa38466a98a5e4ea838916c5) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:55.289150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:55.291756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:55.298648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 330/688] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:54 2022] GET /identity/v3/role_assignments?user.id=2665b33ffa38466a98a5e4ea838916c5&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:55.315236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727c710-3bb0-43ea-9ef4-f23f6aec71b3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:55.351531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:55.363415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:55.364762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:55.365195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:55.365486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:55.366420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:55.366742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:55.368256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:55.395704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 359/689] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 11:50:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:50:55.404548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:55.537020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503958-6083-4502-80cf-07525fd40f5b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:55.575749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:55.576773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:55.577262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:55.577741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:55.607809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:55.652074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:55.653874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:55.760262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 331/690] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:55.769688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:55.885731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686a56d-84dc-4d0f-a3e1-0ba94d8de87e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:55.929823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:55.931032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:55.931521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:55.932036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:55.944819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:55.946629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:55.965712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:55.966644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:55.967928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:55.967928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:55.967928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:55.967928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:55.969908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 360/691] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:55.975983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:56.038037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:56.084086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4d9e-e5fb-48d7-8132-ffcdf8b7205b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:56.122200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:56.122965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.123259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.123545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.124273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:56.124583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:56.125964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f61257-fb8c-4587-ace2-8bee1f46bc31 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:56.138381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 332/692] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:56.143934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:56.219834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:56.349539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128a0bc-59d1-4b9c-9229-7aeeeb11e6ca None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:56.415689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:56.416412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.416692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.416974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.423274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f115a-9b07-40e6-9ed6-31bc6b1404fa None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.423563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f115a-9b07-40e6-9ed6-31bc6b1404fa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.423846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f115a-9b07-40e6-9ed6-31bc6b1404fa None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.425548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 333/693] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:56.429460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:56.430999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:56.435141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.435395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.435684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:56.451199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:56.452195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:56.453080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:56.454989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 361/694] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:56.455944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:56.457124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:56.460518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:56.555042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 334/695] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 06 11:50:56.596929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:56.664080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:56.679580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:56.702305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73977d-4878-44be-a2d8-8e76bc1764d5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:56.726477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f26ac-293c-476f-84cf-da5b6a5391e6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:56.766547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:56.767505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.767862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.768188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.781212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:50:56.788942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:56.789913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:56.790762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 335/696] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:56.799741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:56.841611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:56.842545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.842833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.843157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.843962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:56.844276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:56.845756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:56.859196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 362/697] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:56.867984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:56.913459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff4dcd-753f-45d2-9e45-4e4e6cb03756 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:56.924043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:56.954995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:56.955684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:56.955978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:56.956258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:56.957018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:56.957331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:56.960206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:56.971151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 336/698] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:56.977145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:57.081868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032f7b2-1222-4042-adfd-863809e69018 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.123746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:57.124409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.124699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.124980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.126604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=0a914be52bf745d4b4647f482460e734) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:57.126959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:57.128458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:57.142898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 363/699] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:56 2022] GET /identity/v3/role_assignments?user.id=0a914be52bf745d4b4647f482460e734&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:57.176930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409d52d-f3d9-453e-8487-f6be072748a0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.233007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:57.234168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.234661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.235163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.250144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:57.252166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7e8d739a-1376-49a6-bc17-a270f5566cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:57.277515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:57.278503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:57.279439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:57.279439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:57.279439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:57.279439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:57.281624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 337/700] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:57.287488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:57.328214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:57.338756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c5bc-61e5-4215-b236-75a2457637ca None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.339492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c5bc-61e5-4215-b236-75a2457637ca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.339984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c5bc-61e5-4215-b236-75a2457637ca None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.342197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 338/701] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:57.348418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.349064 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.349717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.370653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:57.372396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:57.405381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:57.476121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0954b1-ea60-4f7c-b12e-3c57bfbcd8df None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.482175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 339/702] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:57.517664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:57.523972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:57.524956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.525460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.525942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.527464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:57.528006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:57.530060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:57.547972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 364/703] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:57.554666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:57.651028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e640f56-a946-4efb-b8a9-ee2643c6e7e6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.666172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:57.691204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5469ac-00cd-45ce-8d35-5ed825ec96e4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.701195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:57.702000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.702288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.702625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.717224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:57.720208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:57.735573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:57.736707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.737223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.737708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:57.744481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:57.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:57.746262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:57.746262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:57.746262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:57.746262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:57.746262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 340/704] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:57.750931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:57.750931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:57.752982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:57.758022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:50:57.770029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:57.771684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:50:57.772638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:57.775424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 365/705] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:50:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:57.781093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:57.892826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc2677-3557-445a-8fd8-f7a2a7317496 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.928638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31a0dd-6cab-428d-9957-a6afea598d0b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:57.955488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:57.956822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:57.958245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:57.959662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:57.961169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:57.963113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:57.965817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:57.991731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 341/706] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:58.000017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:58.000914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.001204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.001490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.001895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:58.002344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:58.002658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:58.004306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d99c8-44e5-4f44-a840-8139a414c493 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:58.027173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:58.040076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 366/707] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:50:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:58.041800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3855b93-9f78-4efe-bd2d-9769a0de51bd None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.042488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3855b93-9f78-4efe-bd2d-9769a0de51bd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.043017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3855b93-9f78-4efe-bd2d-9769a0de51bd None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.048669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 367/708] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:58.049702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3855b93-9f78-4efe-bd2d-9769a0de51bd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:58.085084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:58.128836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376720f-56be-49e5-a82e-559712fcb8f8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:58.169916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3855b93-9f78-4efe-bd2d-9769a0de51bd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:58.172165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adc538-ccbc-43bd-abff-8cc7f09fc609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:58.174140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adc538-ccbc-43bd-abff-8cc7f09fc609 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.174444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adc538-ccbc-43bd-abff-8cc7f09fc609 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.174732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adc538-ccbc-43bd-abff-8cc7f09fc609 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.175497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21adc538-ccbc-43bd-abff-8cc7f09fc609 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:58.177088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21adc538-ccbc-43bd-abff-8cc7f09fc609 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:58.211379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 342/709] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:50:58 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:50:58.213136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.213650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.214083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.214913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:58.215880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.216399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.216874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.244212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:58.246024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:58.357107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:58.370872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:58.370872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:58.406290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 343/710] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:50:58.461726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:58.534525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 368/711] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:50:58.535527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:58.541224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:58.594485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1dcce-01a8-46f1-bb1c-292b2a67526d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:58.635061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea74d6-7ee0-476c-ac3e-9fa1600375aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:58.635811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea74d6-7ee0-476c-ac3e-9fa1600375aa None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.636101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea74d6-7ee0-476c-ac3e-9fa1600375aa None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.636670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea74d6-7ee0-476c-ac3e-9fa1600375aa None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.637412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea74d6-7ee0-476c-ac3e-9fa1600375aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:58.638776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea74d6-7ee0-476c-ac3e-9fa1600375aa None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:58.652279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:58.658020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d9a76-6d6c-4474-94a9-a757d1f205ad None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:58.659194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 344/712] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:50:58 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:50:58.662088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8511afe-b267-4c45-b269-ac9585f12cda None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.662382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8511afe-b267-4c45-b269-ac9585f12cda None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.662669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8511afe-b267-4c45-b269-ac9585f12cda None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.664472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 345/713] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:50:58.670249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.670551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.670862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.688817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:50:58.690074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:50:58.722692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:58.724976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.725483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.726102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.738923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:58.741271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:50:58.767205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:58.768250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:58.769196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:50:58.769196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:50:58.769196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:50:58.769196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:58.772119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 369/714] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:58.776862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 346/715] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:50:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:50:58.779521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:58.819957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:58.914682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:58.938732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf45de-0d49-4de8-8b45-c8ce375d5624 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:58.969033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cea16-a962-456a-9a1f-24f7795cfd69 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:58.981254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:58.982075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:58.982477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:58.982870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:58.983828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:58.984236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:58.985821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.000936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 370/716] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:59.007224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:59.017573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:59.025986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:59.026805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:59.027118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:59.027399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:59.028058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:59.029443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:59.050419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:59.052415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:50:59.053303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:50:59.056545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 347/717] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:58 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:50:59.061933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:59.156795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc988f4-1512-4950-b32f-55962cbabcb2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:59.188555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f973d-9cb0-4469-aa5e-cf5495a685c4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:59.192504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:59.204186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:59.205257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:59.205551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:59.205837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:59.216021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:59.218174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.245729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:59.246908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:59.248908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:59.249410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:59.250328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] RBAC: Adding query filter params (name=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:59.250855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:59.254639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:50:59.256789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:50:59.257710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:50:59.258604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:50:59.267119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 371/718] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:50:59.270459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 348/719] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:50:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:59.272275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:59.276199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:59.400063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:59.419397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5755bf-b044-4cfc-822d-7ab81728b63a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:59.455371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:59.456012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:59.456304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:59.456585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:59.457293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] RBAC: Adding query filter params (type=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:59.457669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:59.459292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.464356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 349/720] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:50:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:59.495262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfcecb-b871-4d3b-8eb8-f8905fd0c223 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:59.554556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:59.555244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:59.555529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:59.555804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:59.556593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:59.556903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:59.558347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd865-9bee-43cf-93b4-9375a8d52623 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.580515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 372/721] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:50:59.586931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:50:59.677587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:50:59.776264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214c484-9466-4ad1-aa90-b29702d9289d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:50:59.815659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:50:59.816325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:50:59.816618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:50:59.816968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:50:59.817849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=b0208c5889a74870b764c7eb954242bb) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:50:59.818219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:50:59.819712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976c973-755e-49ce-b8ee-d21818154d1c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:50:59.826236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 350/722] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:50:59 2022] GET /identity/v3/role_assignments?user.id=b0208c5889a74870b764c7eb954242bb&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:00.457291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab673-a50f-4538-8300-f5c335c6e6f8 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:00.458587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab673-a50f-4538-8300-f5c335c6e6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:00.458815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab673-a50f-4538-8300-f5c335c6e6f8 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:00.460507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 373/723] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:00.466436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:00.466727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:00.467301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:00.484664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:00.485844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:00.565574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 351/724] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:51:00.606023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ab673-a50f-4538-8300-f5c335c6e6f8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:00.749874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ab673-a50f-4538-8300-f5c335c6e6f8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:00.776563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:00.786246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:00.787074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:00.787363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:00.787644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:00.798130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:00.806514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:00.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:00.808439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 374/725] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:00.815939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:00.868507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:00.972827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7eae5-f0b0-4401-9b25-1c314b4c029d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.021016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:01.021694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.021980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.022257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.023033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:01.023353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:01.024730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd75c4-8149-4ac5-b703-c6a80d104352 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:01.038067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 352/726] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:01.043342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.217264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd556b17-6542-4c27-8426-5a604d0fe55f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.234070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77644fc-40d0-4b30-a51c-8686941b837b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.234402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77644fc-40d0-4b30-a51c-8686941b837b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.234653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77644fc-40d0-4b30-a51c-8686941b837b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.236330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 353/727] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:01.243466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.243764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.244052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.266648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:01.267400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.267682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.267962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.271865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:01.273086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:01.286640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:01.288142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:01.316978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:01.318139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:01.319063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:01.319063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:01.319063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:01.319063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:01.320614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 375/728] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:01.324780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.408329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:01.415782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 354/729] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 11:51:01.442148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.462594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:01.492697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766f4d-5c29-4b5c-8874-c405b0c84982 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.528689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:01.529714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.530224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.530721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.531671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:01.532201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:01.533804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843f313-3ee2-4f5e-b013-51fc77e76e3e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:01.546785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 376/730] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:01.548375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38d363-cdf3-4dff-818e-fc83096714a5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.548892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38d363-cdf3-4dff-818e-fc83096714a5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.549385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38d363-cdf3-4dff-818e-fc83096714a5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.551613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 377/731] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:01.562863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.562863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.562863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.578927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:01.580209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:01.584271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285478c-ef22-4712-829f-49335bbabe37 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.624626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:01.625426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.625713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.625994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.641228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:01.649129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:01.650259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:01.651344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 355/732] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:01.652139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.674494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 378/733] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:51:01.675696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.719626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:01.742375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:01.770306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36b41-81fe-4248-bb3f-bda77e3df629 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.801445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e375bca-6cd1-4533-a750-f3e0cd94ce78 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.813118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:01.813920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.814237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.814957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.824805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:01.826272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:01.843579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:01.844577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:01.844754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:01.845665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:51:01.846651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:01.848684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:01.849418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:01.850387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:01.850917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:01.851111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 356/734] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:01.851960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.853193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:01.868254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 379/735] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:01.869544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:01.971281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb75025-d1e0-41f0-9211-134b5d608f02 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:01.981180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:01.995043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a819-19d0-41b2-a614-8347e0f62cff None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.017602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.019139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.019627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.020265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.021510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.023926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.039573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:02.040174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:02.041090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:51:02.042044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:02.044209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.044716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.045209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.046294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:02.046596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 357/736] 10.222.0.41 () {56 vars in 1129 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:02.047091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.047624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.049049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.050917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.080466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 380/737] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 11:51:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:02.081881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.159532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526e94-7596-41fa-90f0-d728acd3d279 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.197253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.197921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.198207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.198534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.210791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.211981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.225761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328e870-8c3f-4899-8510-dec36cf9c367 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:02.229958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:02.230828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:02.231761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:02.231761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:02.231761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:02.231761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:02.235231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 358/738] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:02.236082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.268856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.269552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.270617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.271069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.271999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:02.272565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.273911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.283915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 381/739] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:02.284559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.301262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.312941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.385562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b3985-5e9a-4029-8a14-f3d00f5caf06 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.403709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52ddc4-7554-482f-9141-d2a27d700f9e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.420780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e49861-ed34-4fbf-9afc-f9aa4a83bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.421544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49861-ed34-4fbf-9afc-f9aa4a83bb3b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.421936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49861-ed34-4fbf-9afc-f9aa4a83bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.422307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49861-ed34-4fbf-9afc-f9aa4a83bb3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.443812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.444476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.444760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.445035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.445747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:02.446084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.447503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.464933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 382/740] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:02.464933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.496250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49861-ed34-4fbf-9afc-f9aa4a83bb3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.497722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49861-ed34-4fbf-9afc-f9aa4a83bb3b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.550292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.556193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.583407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ede4e-f341-473f-82fa-835b2de70cf3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.610447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 359/741] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 11:51:02.610447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680fc95-f6c0-4997-bded-9bf2b94deba0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.610447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680fc95-f6c0-4997-bded-9bf2b94deba0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.610447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680fc95-f6c0-4997-bded-9bf2b94deba0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.610447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 360/742] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:02.610447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680fc95-f6c0-4997-bded-9bf2b94deba0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.627636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.628306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.628595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.628873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.629845 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:02.630176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.631584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.637913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 383/743] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:02.638648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.720269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680fc95-f6c0-4997-bded-9bf2b94deba0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.743795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b277-9b1d-493e-909a-70d9fa4ed305 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.755929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.756739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.757029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.757314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.764486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.765944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.777846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.778622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.778935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.779208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-df08356c-a619-4b1f-be3f-19b13af66c39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:02.781002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:02.781942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:02.782906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 361/744] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:02.783806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff974-ca13-4d3a-ba43-31f47ed70fc7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.784377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff974-ca13-4d3a-ba43-31f47ed70fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.784377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff974-ca13-4d3a-ba43-31f47ed70fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.787051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.792692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.794167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.802292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-169ff974-ca13-4d3a-ba43-31f47ed70fc7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:02.803524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-169ff974-ca13-4d3a-ba43-31f47ed70fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:02.811065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:02.812117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:02.813025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:02.813025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:02.813025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:02.813025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:02.813025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 384/745] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:02.813385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.813670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:02.874374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 362/746] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:51:02.875733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5629a-abf6-4016-837d-1e0e576b80fa None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.876026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5629a-abf6-4016-837d-1e0e576b80fa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.876308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5629a-abf6-4016-837d-1e0e576b80fa None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.877570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 363/747] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:02.878262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5629a-abf6-4016-837d-1e0e576b80fa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.926411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228b2b7-68cf-4ad1-8cd1-f929072713f5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:02.959910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:02.960538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:02.960831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:02.961113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:02.961929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] RBAC: Adding query filter params (interface=public, service_id=2e32568bf96d49a3a7088762cf34ab03, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:02.962241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:02.963670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:02.971485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 385/748] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/endpoints?service_id=2e32568bf96d49a3a7088762cf34ab03&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:02.972267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:02.986032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5629a-abf6-4016-837d-1e0e576b80fa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.022908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.023584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.023879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.024162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.025004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:03.025325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.026770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11002ff-2094-4d4a-924e-ab6bca8b6b9b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.035236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.044778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 364/749] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:03.046025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.046321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.046600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.051380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.064250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:03.065438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:03.084367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ba3cc-c272-4c09-9f39-d69f5d8e5ae6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.116687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.117337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.117624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.117902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.118609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:03.118956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.120341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.132501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 386/750] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:03.133268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.135098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 365/751] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 11:51:03.135846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.236292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3291d4-6fed-495a-9e8c-d04cca88ad7a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.238279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d49e-bbf2-4204-9fd3-6df89a0e6595 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.249634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.272674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.273892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.274731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.275060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.275363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.276085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.277474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.278755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.279454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.279742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.280087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.280763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.282179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.287886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 366/752] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/services => generated 544 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:03.288763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.302866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 387/753] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:03 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:03.302866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.453640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fff0a1-15cc-46df-bd13-9e4dd619ff2b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.455175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f4257-0507-4978-9a91-654c1183bfa3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.497795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.498520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.498878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.499602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.509517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.509725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.511138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.511248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.511691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.512144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:03.528563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:03.529500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 388/754] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:03.533447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.569133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.595761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.610334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.612314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.666648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2d74c-2469-47ac-a5f2-fceae59699db None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.702872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.703944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.704385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.704814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.718602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:03.727612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:03.728628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:03.729488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:03.736981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 389/755] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:03.738261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.754804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 367/756] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:03.759236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.847951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.865673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45340bd-cbda-4998-955d-e07fb210f49a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.886132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:03.892879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54d0b4-87c1-44ee-a05e-a14833030d2e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:03.919835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.921004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.921297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.921583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.922376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:03.922811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.924767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.937307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:03.938332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:03.938826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:03.939358 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:03.940380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:03.946048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:03.946175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 390/757] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:03.946175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:03.947735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:03.962247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 368/758] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:03.963626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.079993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099004b-f473-434d-8de7-2dd2c98af70f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.080198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce750f7-33c4-4f9f-a4be-f72e5512044b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.119154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.120268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.120780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.121366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.122401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=0a914be52bf745d4b4647f482460e734) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:04.123178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.124568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.125148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.125369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.125656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.125994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.139268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 369/759] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/role_assignments?user.id=0a914be52bf745d4b4647f482460e734&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:04.139478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.140781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.140911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.153236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:04.164991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:04.165959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:04.166895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:04.166895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:04.166895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:04.166895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:04.166895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 391/760] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:04.172252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.201982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:04.299189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460daa7-68af-448e-b35c-ce15345b4238 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.343127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.344384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.344680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.345114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.345869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:04.346184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.347652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.361945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 392/761] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:04.369386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21943-d3b2-45ba-a260-78bfa1a3a4f0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.370736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.467372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:04.468180 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.469005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.469277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.469557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.487342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec336bf4-d5d2-4e89-8754-464f77eacf47 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.499994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.501475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:04.534884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:04.536063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:04.537120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:04.537120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:04.537120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:04.537120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:04.537120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 370/762] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:04.547167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.547798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.548526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.549027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.549565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.558397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.560347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:04.576997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:04.578354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:04.579284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:04.581846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 393/763] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:04.592301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.663215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:04.672725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9074c2-0bb7-4dde-89ca-6b9142d118cf None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.708790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.709467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.709760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.710046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.710775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:04.711119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.713105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.727066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 371/764] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:04.740137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.775404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbab8-7023-4e10-b047-d5451d5bcf10 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.796580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:04.812808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.813393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.813678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.813957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.814742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:04.815081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.817087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.834334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 394/765] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:04.835602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:04.874189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196c07-a3bc-4e6c-884b-dce07d85abe4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:04.910943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:04.913571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:04.914410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:04.915237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:04.924358 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:04.926624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:04.947072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:04.956254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:04.957869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:04.958863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:04.959959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 372/766] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:04.965449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:05.001990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a554-93af-4bfb-ac19-8a6b28f643fc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:05.075430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:05.076013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.076299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.076580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.077421 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=5b62f565650b42f5b4a1bc2f8b39b439) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:05.077736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:05.079230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:05.096762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 395/767] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/role_assignments?user.id=5b62f565650b42f5b4a1bc2f8b39b439&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:05.137221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e43914-e53e-4cda-b972-a9f5fd8d31eb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:05.174674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:05.175383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.175670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.176535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.177319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:05.177633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:05.179174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:05.199946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 373/768] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:51:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:05.205541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:05.275005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:05.337235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb0480-fb7a-41c6-b84a-edba1c000ad1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:05.375023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:05.375835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.376119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.376396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.389144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:05.390649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:05.409883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:05.411422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:05.412323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:05.412323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:05.412323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:05.412323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:05.412990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 396/769] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:05.418696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:05.483056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:05.537888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9669b-1846-4c2f-899f-09cde4125185 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:05.581859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:05.582858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.583224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.583527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.584253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:05.584563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:05.585943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:05.598887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 374/770] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:05.604236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:05.754567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:05.766702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746481b-8066-49e6-8e15-36d39e8d216b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:05.818987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:05.819796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.820081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.820360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.833855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:05.835411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-09967b42-0300-43ca-b4a0-f5cf73ee8385 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:05.859165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:05.860080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:05.860967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 397/771] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:05.868103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:05.881939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b93fd-1a5c-46e9-af3f-7e3f9e302382 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.882225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b93fd-1a5c-46e9-af3f-7e3f9e302382 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.882514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b93fd-1a5c-46e9-af3f-7e3f9e302382 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.883986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 398/772] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:05.890936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ec679-b277-4222-aa9e-d9918fda4eb1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:05.891232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ec679-b277-4222-aa9e-d9918fda4eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:05.891979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ec679-b277-4222-aa9e-d9918fda4eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:05.924576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-345ec679-b277-4222-aa9e-d9918fda4eb1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:05.925828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-345ec679-b277-4222-aa9e-d9918fda4eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:06.047972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840a5d8-cdc6-48ee-a25d-864b2550b58c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.065226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 399/773] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:06.076570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:06.100780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.101450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.101737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.102017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.102801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:06.107194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.108558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210720ae-6e6d-40d9-9005-d7d414a9490f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.108695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f69f63-baff-4882-9e2b-781b5475ee49 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.108849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210720ae-6e6d-40d9-9005-d7d414a9490f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.109135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210720ae-6e6d-40d9-9005-d7d414a9490f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.110605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 400/774] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:06.119721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210720ae-6e6d-40d9-9005-d7d414a9490f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.125049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 375/775] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:06.126543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.127065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.127568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.145286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:06.146791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:06.219273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:06.223276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 376/776] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:06.224247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.236715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210720ae-6e6d-40d9-9005-d7d414a9490f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.277381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.278201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.278497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.278785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] PATH_INFO: `/v3/services/network` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.279479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.280901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:06.302722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:06.303817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:51:06.304657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:06.305454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 401/777] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:06.310862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.349198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26121eda-97e6-4269-b95f-29ebddf70694 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.350950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:06.391967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.393084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.393620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.393907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/b0208c5889a74870b764c7eb954242bb/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.396790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=b0208c5889a74870b764c7eb954242bb, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.398520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.421826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 377/778] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:06 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/b0208c5889a74870b764c7eb954242bb/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:51:06.423408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.438469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6277b-f47b-4246-8819-a28e1b32b019 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.481765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.482787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.483202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.483604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.484409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.485957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:06.501194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:06.503245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:06.505052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 402/779] 10.222.0.41 () {56 vars in 1129 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:06.514872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.529166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:06.587799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e024d5-e047-4076-85bb-1f7b93e39afb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.631489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd81d5d-fc51-45ad-a238-3e9f3a1aa855 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.639045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.639903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.640337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.640980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.641544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:06.642067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.643960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.649913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 378/780] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:06.655576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.682635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.683524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.683932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.684287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.685485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:06.685607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.687123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.692825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 403/781] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:06.693509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:06.698736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.792044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faa1c4-3eb8-4cbc-9f4a-bbc7ea7426f1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.832818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.833516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.833805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.834088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.835649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] RBAC: Adding query filter params (type=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:06.835971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.837411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.843735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:06.845609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 379/782] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:06.852589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.888100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286a4f6-9d7c-4762-8afa-4e60fd787b3b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.934003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:06.934591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.934907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.935198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.935912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:06.936226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:06.938102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d47c8-2874-4267-b2be-20d0272bcda4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:06.946288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 404/783] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:06.947813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a9ab-9291-4389-9f49-23094d64026f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:06.948109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a9ab-9291-4389-9f49-23094d64026f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:06.948409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a9ab-9291-4389-9f49-23094d64026f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:06.954726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 405/784] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:06.955570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a9ab-9291-4389-9f49-23094d64026f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:06.967963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7385e0-a15d-4d6e-be1d-1c65bc822b1e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:06.971273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:07.009466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.010546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.011116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.011775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.012777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] RBAC: Adding query filter params (interface=public, service_id=3487e1009afb44a3ad8f442071e9621b, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:07.013301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:07.014992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26272936-4942-433e-9907-f3a29649ef00 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:07.032667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 380/785] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:06 2022] GET /identity/v3/endpoints?service_id=3487e1009afb44a3ad8f442071e9621b&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:07.034308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.034829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.036072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.052959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:07.054904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:07.083972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a9ab-9291-4389-9f49-23094d64026f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:07.120438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.121498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.122052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.122527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.123558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:07.125160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:07.146863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:07.147283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 381/786] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:07.148321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:07.154767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 406/787] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:07.178055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:07.268844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:07.288549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b453439-f9fb-43a9-bf75-363bbf1ad0ca None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:07.331039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0d599-cc45-48bc-9488-ec4abed43aaf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:07.346713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433947f-f597-4123-8f09-24574e89874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.347420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433947f-f597-4123-8f09-24574e89874d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.347806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433947f-f597-4123-8f09-24574e89874d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.348139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433947f-f597-4123-8f09-24574e89874d None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.348835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433947f-f597-4123-8f09-24574e89874d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:07.350506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433947f-f597-4123-8f09-24574e89874d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:07.362460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 382/788] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:07 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:07.389053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.389877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.390160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.390433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.403454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:07.411794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:07.412842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:07.414117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 407/789] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:07.419619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433947f-f597-4123-8f09-24574e89874d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:07.535339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433947f-f597-4123-8f09-24574e89874d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:07.574682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.575593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.575883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.576161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.576921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:07.577227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:07.578623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:07.589301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 383/790] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:07.595525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:07.662595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:07.715519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01bf11-5738-4dac-8b39-26a067524bb1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:07.756880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.757814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.758104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.758382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.771627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:07.773135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:07.792708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:07.794945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:07.797318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:07.797318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:07.797318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:07.797318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:07.797318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 408/791] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:07.802929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:07.819212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:07.906813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e782681-4729-47d3-b7dd-3dbb811f0cff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:07.942762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:07.943453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:07.943737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:07.944078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:07.944798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:07.945102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:07.946557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:07.960777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 384/792] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:07.966155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.074207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda1f14-ce9d-40e2-861b-06b905f353a4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.081477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:08.109908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.110714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.111027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.111305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.119730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:08.121196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-53fae5ba-5616-4eb0-bca4-3580e7d7d707 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:08.135716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:08.136697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:08.137615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 409/793] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:08.142813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.171674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990fb28-87e2-497d-a2e3-a5ac23284a37 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.172024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990fb28-87e2-497d-a2e3-a5ac23284a37 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.172267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990fb28-87e2-497d-a2e3-a5ac23284a37 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.174895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 410/794] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:08.179846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.180136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.180414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.215877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:08.217353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:08.219246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:08.257296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd1b78-eed7-4b13-9470-8523c6982a1b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.295504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.298863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.298863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.298863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.298863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:08.298863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:08.299898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:08.321573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 385/795] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:08.326964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.406903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 411/796] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:08.433097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9caa138-f284-4ce4-aa23-b7d5beaf9479 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.444471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.463224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:08.472233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.473072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.474008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.474324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.485885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:08.487620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:08.508340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:08.509268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:08.510140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:08.510140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:08.510140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:08.510140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:08.510140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 386/797] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:08.515249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.629526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f800d92-ca9f-49bd-864d-5ca2f011859b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.666419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.667309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.667601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.667889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.668599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:08.668904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:08.670283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:08.682625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:08.687186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 387/798] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:08.689968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.694190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fda59-9e2e-4188-b580-de2a52251ad6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.747607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:08.771577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.772399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.772688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.772971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.792274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:08.798225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b54b-f817-487c-b41c-cec165007843 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:08.808309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:08.809255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:51:08.810104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:08.814348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 412/799] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:08.819819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.851388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.852543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.853027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.853540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.872038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:08.873855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:08.902947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:08.903852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:08.904818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:08.907396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 388/800] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:08.913148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:08.925345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99788b11-c749-4330-ac3e-79a2a5d83ee8 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:08.968677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:08.969364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:08.969652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:08.969937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:08.970707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:08.971099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:08.972548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.006690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 413/801] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:09.009052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.054440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.130426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f583d-5b85-4444-b8c7-0b96359eca07 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.134437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.198991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6c1a2-fd54-40c6-91f1-5c8dbcf8344d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.203384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.204054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.204341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.204623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.205413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:09.205723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.207264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.223043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 389/802] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:09.228762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.244080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.244874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.245159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.245434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.256842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.258321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:09.277523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:09.278470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:09.279375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:09.279375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:09.279375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:09.279375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:09.283238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 414/803] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:09.284713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.339126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e39a6e-9a05-4167-a792-1f5ffff90644 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.359606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.390619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20fed4-dfc0-42c5-8936-2f8a4e5102ca None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.426570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.427289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.427571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.427849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.428586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:09.428901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.430417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b6fb2-7760-41ad-b802-bf9b2ee49f37 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.431539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c10b73-b7b8-4b54-b141-064f436bbfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.432449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c10b73-b7b8-4b54-b141-064f436bbfe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.432739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c10b73-b7b8-4b54-b141-064f436bbfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.433018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c10b73-b7b8-4b54-b141-064f436bbfe4 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/0a914be52bf745d4b4647f482460e734/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.439692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c10b73-b7b8-4b54-b141-064f436bbfe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=0a914be52bf745d4b4647f482460e734, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.442111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c10b73-b7b8-4b54-b141-064f436bbfe4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.446921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 415/804] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:09.448787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f27c7-8d71-4673-9165-c82d54d1ff98 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.449487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f27c7-8d71-4673-9165-c82d54d1ff98 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.449744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f27c7-8d71-4673-9165-c82d54d1ff98 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.451585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 416/805] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:09.458431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f27c7-8d71-4673-9165-c82d54d1ff98 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.464226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 390/806] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:09 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/0a914be52bf745d4b4647f482460e734/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:51:09.465607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.465898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.466176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.487805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:09.489960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:09.555263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.566968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 391/807] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:09.592478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.609590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f27c7-8d71-4673-9165-c82d54d1ff98 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.651177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.659507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.661754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.662518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.663024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.673172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.676115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:09.698019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:09.699559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:09.701475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 417/808] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:09.705937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.714154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7418-b347-4404-85e5-ef8f3da9b327 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.762111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.763467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.763944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.764757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.779995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:09.790594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:09.791642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:09.792482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:09.795440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 392/809] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:09.802070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.850252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fceb3d-db59-4a5a-b6c9-a91894968fbe None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.857775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.893749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.894833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.895559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.895945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.896945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:09.897357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.898945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:09.920394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 418/810] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:09.926318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:09.947917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b682fb6-698f-4205-9111-3ea2b2c87ae3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:09.980859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:09.986731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:09.987456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:09.987723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:09.988086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:09.988846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:09.989151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:09.990775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db2996-428e-4b5c-af99-15979faf577e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.000100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 393/811] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:10.001404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a17772-83d7-48cc-9863-456a97a8c055 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.001696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a17772-83d7-48cc-9863-456a97a8c055 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.001979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a17772-83d7-48cc-9863-456a97a8c055 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.003499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 394/812] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:10.009331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.009618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.009897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.037025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:10.038576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:10.052993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1609-3849-4b94-9f47-b61f6a944707 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.096360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.097825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.098109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.098557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.111773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.113512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.123047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 395/813] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:10.124682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:10.134711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:10.135650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:10.136566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:10.136566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:10.136566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:10.136566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:10.136566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 419/814] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:10.142007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.147194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.247724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df1a94-9376-45ba-b67a-1194c078740c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.258042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.261811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a3283-5104-46ce-97cc-95d097338a70 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.285158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.286305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.286756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.287194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.298665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.300332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.301686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.303001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.303500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.303887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.304727 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:10.305137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.306660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.324082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 420/815] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:10.324082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:10.326000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:10.327029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:10.328107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:10.328107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:10.328107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:10.328107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:10.329909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 396/816] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:10.330820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.392311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.435287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07125ac-2115-4f37-8484-97f57083b308 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.442496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e252c2d-091d-4e9d-806c-99fa5e519025 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.471965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.473340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.473753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.474196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] PATH_INFO: `/v3/services/network` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.475002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.476750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.481709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.482753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.483230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.483651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:10.490088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:10.491240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 11:51:10.492254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:10.494070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 421/817] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:10.494787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.496447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.500173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:10.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:10.512751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:10.513654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:10.521214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 397/818] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:10.522321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.530486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.622812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44d47-59dc-459b-8ea0-9df0c855bb6a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.642038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120fd33b-fab9-4153-b00a-9ae2d264f21c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.657201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.658716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.659434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.660260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.660618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.661341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:10.661655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.663065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.676516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 422/819] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:10.677517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.686020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.686914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.687255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.687543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.688241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] RBAC: Adding query filter params (name=network) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:10.688549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.689904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4f5d4-cc1c-4820-88ec-0de0fb574b99 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.698990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 398/820] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:10.698990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192853-8845-455b-b27d-1294894b6a0e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.698990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192853-8845-455b-b27d-1294894b6a0e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.698990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192853-8845-455b-b27d-1294894b6a0e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.698990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 399/821] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:10.698990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192853-8845-455b-b27d-1294894b6a0e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.776193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.793631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a862dca-5de9-4c07-a52c-dc3cc2d33be5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.810225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192853-8845-455b-b27d-1294894b6a0e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.828608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.829565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.829847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.830634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:10.830983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.832565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.844620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 423/822] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:10.845430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:10.851973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:10.852965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.853391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.853901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.862090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:10.864156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-67e44cee-721e-47b6-84af-fd52499f7563 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:10.878967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:10.880045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 400/823] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:10.885847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:10.886728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:10.901053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:10.902340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:10.937461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:10.986877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 401/824] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:10.987094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f4bd6-0b76-4681-8c52-6a4682c310be None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:10.987189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.038019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.038604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.038908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.039189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.039892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] RBAC: Adding query filter params (type=network) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:11.040197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.041578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.046541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 424/825] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:11.047359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.062036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:11.114003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c06ed-bb5e-4117-8787-76233d6401a7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.148963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.149847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.150130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.150413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/5b62f565650b42f5b4a1bc2f8b39b439/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.159364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fd1ef-eea9-4d05-983a-9ee7784b241c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.166088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=5b62f565650b42f5b4a1bc2f8b39b439, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.167582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.181434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 402/826] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:10 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/5b62f565650b42f5b4a1bc2f8b39b439/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:51:11.182458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.209073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.209814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:11.211137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.211606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.212175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.213303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:11.213844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.216093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.237832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 425/827] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:51:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:11.239649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.321702 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba144673-7c82-419f-ae55-fb3b69df93a6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.356733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:11.366911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.368152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.368634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.369102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.369969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.371658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:11.389648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:11.390701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:11.392060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:11.396449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 403/828] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:11.396449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.409656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90c0e3-ee62-44a7-af6f-7dc495fd920d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.453563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.454577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.455097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.455553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.456461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.458700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.482338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 426/829] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:11.490868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.556568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:11.624219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142107c-717e-47ab-bff8-55082d2b92af None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.636936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853fe90-506d-4f70-999e-178c0bc9bdf2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.651260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:11.668167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.668882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.669171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.669483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.670263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:11.670576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.672052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.689205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 427/830] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:11.695097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:11.724633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.725653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.726162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.726647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.813691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.815568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.844425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f04e99-6ea8-46f8-9c3c-1c8021170a1d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:11.899210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:11.899803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:11.900089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:11.900368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:11.901079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:11.901389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:11.902778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:11.909649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 428/831] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:11.935054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:11.959428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 404/832] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:11.965276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.031068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:12.098550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef5dcd-639c-41d5-be50-5e8b9b5babc7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.136801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.137972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.138483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.138990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.151525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:12.153203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-71324560-c3de-4c68-bf32-6fed99c164b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:12.171578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:12.172716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:12.173812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:12.173812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:12.173812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:12.173812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:12.176403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 429/833] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:12.182887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.235666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef015-ed1c-49a7-9033-3f08b247d2b6 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.235963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef015-ed1c-49a7-9033-3f08b247d2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.236251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef015-ed1c-49a7-9033-3f08b247d2b6 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.238485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 430/834] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:12.243968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.244263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.244551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.262610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:12.263853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:12.299699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10481f77-1809-47bc-a021-a585493c5e3d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.335535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.337491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.337934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.338367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.339724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:12.340193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:12.342436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:12.349521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 431/835] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:12.360475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 405/836] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:12.366544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.373528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.407296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:12.428800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:12.495807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03b22-ad73-442a-a452-0005b68f8e26 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.522908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d9921-36f9-4142-8797-17f38f8d55a5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.532747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.533631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.533987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.534338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.546330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:12.554200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:12.555213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:12.556232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 406/837] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:12.563019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.567542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.568621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.568972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.569328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.579430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:12.581202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:12.603274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:12.604238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:12.605539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 432/838] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:12.610403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.698393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a05a3-2802-4b85-9618-aa89b97d1935 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.700484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:12.740739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:12.744541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.745860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.746383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.746970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.747925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:12.748438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:12.750263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:12.757873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14102165-858f-4a36-bc40-a0a72927e51e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.761559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 407/839] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:12.768636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.799183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.800983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.801754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.802501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.803507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:12.804558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:12.807065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:12.826987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 433/840] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:12.833413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:12.898174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90125c-cb35-4f5a-910d-ea3f0e67f3b2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.941240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:12.942871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:12.943366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:12.943888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:12.955714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:12.957568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:12.972876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dcc4a-4fdd-427a-8f87-042795e07f72 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:12.978035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:12.979274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:12.980258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:12.980258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:12.980258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:12.980258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:12.986969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 408/841] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:12.988741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:13.009826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:13.024504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:13.025178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.025467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.025750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.026595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=b0208c5889a74870b764c7eb954242bb) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:13.026944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:13.028425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:13.042892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 434/842] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/role_assignments?user.id=b0208c5889a74870b764c7eb954242bb&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:13.129076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b75238-8a2c-40a7-9eba-71a818661c43 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:13.175173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:13.176443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.176950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.177442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.178566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:13.179892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:13.181617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:13.196799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 409/843] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:13.203397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:13.227877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:13.327890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78678f6-ffea-448f-8bd8-33b938d6fcbd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:13.368342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:13.369628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.369923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.370274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.382251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:13.383753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:13.402686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:13.403767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:13.404704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 435/844] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:13.412297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:13.500428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:13.528175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d81c5-cc98-41fb-8d7f-acbece014eb3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:13.565038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:13.566635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.567447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.567962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.569003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:13.569533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:13.571203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8b752-5c58-4fb6-8bb3-5620b9bf30ea None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:13.592008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 410/845] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 11:51:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:13.600114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:13.676025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:13.708710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fa91e-fea9-4f9d-9407-ea2605d6f4db None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:13.746175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:13.746852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.747151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.747433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.797056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a006-dc34-40c8-9614-a58bdd218df5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.797444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a006-dc34-40c8-9614-a58bdd218df5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.797667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a006-dc34-40c8-9614-a58bdd218df5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.799016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 411/846] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:13.805305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:13.805597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:13.805879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:13.824360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:13.826083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:13.832480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:13.833852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:13.911118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:13.946917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 412/847] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:13.963925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 436/848] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:13.971208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:13.975971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.043228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.080876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94439c-f679-4a45-8336-38605f8e8ac0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.085088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e1cc5-ec0a-49dd-86d8-fa1e2bee6fc2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.116414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.117232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.117518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.117797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.129810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:14.132067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.133473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.134071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.134667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:14.138078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:14.139906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:51:14.141553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:14.146954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 413/849] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:51:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:14.148148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.149752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.151448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:14.176417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:14.177407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:14.178740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:14.178740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:14.178740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:14.178740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:14.178740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 437/850] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:14.183612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.218726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.260833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9566956-4b91-448d-9e6c-0085c74f01b5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.289761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.305098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.306013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.306568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.306888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.307638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:14.307957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.309324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333e421-ab07-4a5e-9cdb-9869410d49ed None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.318626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 414/851] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:14.324119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2000e7-2b81-402e-8414-312bdec5f160 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.324417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2000e7-2b81-402e-8414-312bdec5f160 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.324699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2000e7-2b81-402e-8414-312bdec5f160 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.324869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d202065-7d89-4e6a-8b77-00496ce30fd3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.326107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 415/852] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:14.330035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2000e7-2b81-402e-8414-312bdec5f160 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.364234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.364975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.365319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.365538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.366262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:14.366571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.367977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c168091-365c-4a69-8bdb-bfe4b18361d6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.384227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 438/853] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:14.384227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.384227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.384227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.399023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:14.400208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:14.447465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2000e7-2b81-402e-8414-312bdec5f160 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.478891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 439/854] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:14.478891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.489009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.498342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.499728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.500190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.500629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.512848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.514312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:14.531949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:14.532914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:14.533811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:14.533811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:14.533811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:14.533811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:14.533811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 416/855] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:14.534092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.579007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.589837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aca029-9293-4dc4-9925-527e6f38b24c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.624853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.625573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.625862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.626143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.633671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.635171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.648355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd782e85-b7ab-4b4a-851c-b8f59c6009fc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:14.649314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:14.650172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:14.651104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 440/856] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:14.651591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.691929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.692604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.692893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.693174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.693892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.695715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.722871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 417/857] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:14 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:14.722871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.736492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.779048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6be05-6061-445b-8233-791af66dc96e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.842113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.842789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.843104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.843575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.844316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:14.844626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.846039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.854106 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:14.861079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39f37-1da1-46c3-82b9-edea2ea86213 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:14.865098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 441/858] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:14.866389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.902515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:14.903227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:14.903523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:14.903811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:14.904620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:14.904939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:14.906424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:14.921261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 418/859] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:14.928082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:14.986100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497cd9-cda4-4b9d-8f09-3ac24c5f2978 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.027274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:15.035798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.036800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.037081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.037423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.045478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.047076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.058516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6d4a3-c6d7-4b89-84ac-589dc521e5c2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:15.064307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:15.065260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:15.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:15.071141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 442/860] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:15.072604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:15.106650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.107578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.108028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.108500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.109506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=0a914be52bf745d4b4647f482460e734) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:15.110005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.111635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.118988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 419/861] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:14 2022] GET /identity/v3/role_assignments?user.id=0a914be52bf745d4b4647f482460e734&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:15.192963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b484e6-23c5-4fd0-a09b-15c149649d39 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.233694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.234385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.234669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.234988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.235842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:15.236148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.237663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18149-98ec-4c8f-8faa-a187dd8e722f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.259293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 443/862] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:15.264944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:15.294532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:15.383470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4cf04-922a-42b2-a42e-041526d3f5be None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.420353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.421009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.421295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.421574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.494396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.495818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.513307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e665c12-8319-439d-a19b-2e10d6c45ff9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.513613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e665c12-8319-439d-a19b-2e10d6c45ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.513853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e665c12-8319-439d-a19b-2e10d6c45ff9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.516640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 444/863] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:15.524179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.524467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.524750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.532122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:15.545021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:15.546517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:15.599234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:15.620507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 445/864] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:15.635554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 420/865] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:15.642867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:15.645203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:15.749229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6d3eb-f132-4f2b-8e6b-ea954d3b6a22 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.760077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb6c12-58cc-47e0-a76b-bb5ae45eb42a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.785423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.786229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.786519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.786802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.791148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:15.798069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.799582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.806829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.808423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.808936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.809850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.811032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.812639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:15.819210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:15.820084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:15.820968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:15.820968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:15.820968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:15.820968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:15.825249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 446/866] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:15.826090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:15.829275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:15.830182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:15.831055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:15.840296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 421/867] 10.222.0.41 () {56 vars in 1132 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:15.840943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:15.866247 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:15.939961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b22ddf-24e2-4637-ab5f-10394711d102 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.979159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:15.979851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:15.980115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:15.980395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:15.981121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:15.981430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:15.982825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:15.987072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca37f8-3447-4090-9d43-71b54d353c7c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:15.999897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 447/868] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:16.001232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:16.039324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:16.041053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:16.041553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:16.042040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:16.043244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:16.043766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:16.046293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:16.057843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 422/869] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:16.059186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:16.113462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:16.127454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:16.140827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a2aeb-d8aa-4cdc-a7f5-137a82b5865c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:16.178150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:16.178698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7e46c-7a4b-459d-87bc-891956542a86 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:16.178973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:16.179260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:16.179542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:16.186421 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:16.188304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:16.207379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:16.208333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:16.209622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:16.213407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 448/870] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:16.214488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:16.230595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:16.231167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:16.231495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:16.231781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:16.232486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:16.232792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:16.234566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:16.247434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 423/871] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:16.353153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d4993-b2e2-46dc-b7c4-c6b14b2dc7ac None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:16.395846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:16.396905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:16.397441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:16.397930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:16.398980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:16.399507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:16.401187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b37f1-530a-440d-afbc-a2d21dc09b64 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:16.415560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 449/872] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:16.424053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:16.542005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74581584-a6be-436a-a3bc-a349eb4fad45 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:16.558160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:16.581056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:16.581936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:16.582252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:16.582598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:16.583653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=5b62f565650b42f5b4a1bc2f8b39b439) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:16.584199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:16.586514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7827ce-5d69-4670-a7d1-3d747bf7b242 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:16.595308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 424/873] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:16 2022] GET /identity/v3/role_assignments?user.id=5b62f565650b42f5b4a1bc2f8b39b439&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:17.145109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dcba0-453d-41e7-8a27-285a8b80f9ba None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.146212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dcba0-453d-41e7-8a27-285a8b80f9ba None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.146730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dcba0-453d-41e7-8a27-285a8b80f9ba None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.148624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 450/874] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:17.154681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd174-7b48-45ae-a795-20f34509244b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.155010 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd174-7b48-45ae-a795-20f34509244b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.155542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd174-7b48-45ae-a795-20f34509244b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.173482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-abecd174-7b48-45ae-a795-20f34509244b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:17.175693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-abecd174-7b48-45ae-a795-20f34509244b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:17.250222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 425/875] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:17.273324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dcba0-453d-41e7-8a27-285a8b80f9ba None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.286703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:17.384201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dcba0-453d-41e7-8a27-285a8b80f9ba None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:17.423400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:17.424206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.424499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.424779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.425499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:17.426929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-69bebbee-af95-4ac4-9fe3-d3d5c6e3f86b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:17.439590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:17.441244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:17.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 451/876] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:17.447958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecd174-7b48-45ae-a795-20f34509244b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.555143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecd174-7b48-45ae-a795-20f34509244b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:17.565219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:17.570599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934bce5-ab00-409b-bbfd-c2241b1caf44 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.571195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934bce5-ab00-409b-bbfd-c2241b1caf44 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.571709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934bce5-ab00-409b-bbfd-c2241b1caf44 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.574564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 452/877] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:17.578768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.579315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.579827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.593995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:17.594664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.594986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.595272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.595980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:17.596290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:17.598184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:17.599539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:17.601029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:17.603399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 426/878] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:17.608808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.694861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 453/879] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:17.710159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.713967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:17.728787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de50ac0-f9e0-4e04-8a89-2af5435f49f3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:17.763978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:17.764648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.765305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.765305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.765895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:17.766208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:17.767599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:17.773437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 427/880] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:17.778965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.815643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:17.816705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca237b3-9be5-47b8-bb31-322695393f03 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:17.853548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:17.854747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.855292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.855786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.856643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:17.858256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:17.874252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:17.875405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:17.876269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:17.884493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 454/881] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:17.886080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.893108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb962-fb19-4c5a-b158-f1042a8b38d0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:17.931819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:17.932423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:17.932715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:17.932994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:17.933792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] RBAC: Adding query filter params (interface=public, service_id=4fb3b41a134d42458423a1af37889d40, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:17.934106 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:17.935623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:17.946782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 428/882] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/endpoints?service_id=4fb3b41a134d42458423a1af37889d40&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:17.947876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:17.961042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:17.998097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc695be8-3d76-4830-a5e7-93b39634ae0d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:18.034104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:18.035516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:18.036335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:18.036335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:18.037144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:18.037575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:18.039092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:18.048936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 455/883] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:18.050526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:18.061835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:18.067133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189a026-d997-4b3e-8cfe-9849f8e1dcf1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:18.105812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6345f3-93ef-40f0-9f88-24840735465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:18.106727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6345f3-93ef-40f0-9f88-24840735465b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:18.107195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6345f3-93ef-40f0-9f88-24840735465b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:18.107646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6345f3-93ef-40f0-9f88-24840735465b None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:18.108568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6345f3-93ef-40f0-9f88-24840735465b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:18.110156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6345f3-93ef-40f0-9f88-24840735465b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:18.116837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 429/884] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:17 2022] GET /identity/v3/services => generated 1011 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 11:51:18.168488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b55f47-a14e-4a24-a25f-360b286ca4dd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:18.202750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:18.209651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:18.210772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:18.211397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:18.211919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:18.212882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:18.213426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:18.215148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3409c7-566d-40b2-9c8e-894d1a36d2df None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:18.221548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 456/885] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:18.554398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc716e7-8872-425c-a672-21c2b21c20d1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:18.554780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc716e7-8872-425c-a672-21c2b21c20d1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:18.554994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc716e7-8872-425c-a672-21c2b21c20d1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:18.556570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 430/886] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:18.561966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fe63-e25c-415e-a8bb-f3b8359028dc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:18.562258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fe63-e25c-415e-a8bb-f3b8359028dc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:18.562536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fe63-e25c-415e-a8bb-f3b8359028dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:18.580754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d125fe63-e25c-415e-a8bb-f3b8359028dc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:18.582921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d125fe63-e25c-415e-a8bb-f3b8359028dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:18.662090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 457/887] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:18.685123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc716e7-8872-425c-a672-21c2b21c20d1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:18.801681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc716e7-8872-425c-a672-21c2b21c20d1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:18.838014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a42ca-3ebf-496c-b6d2-8ed09cc0f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:18.838602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a42ca-3ebf-496c-b6d2-8ed09cc0f792 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:18.838911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a42ca-3ebf-496c-b6d2-8ed09cc0f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:18.839195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a42ca-3ebf-496c-b6d2-8ed09cc0f792 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:18.839918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a42ca-3ebf-496c-b6d2-8ed09cc0f792 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:18.841590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a42ca-3ebf-496c-b6d2-8ed09cc0f792 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:18.857101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 431/888] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:18 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:19.080850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99144-a188-4382-a212-1ab1d36d6eab None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:19.081242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99144-a188-4382-a212-1ab1d36d6eab None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:19.081687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99144-a188-4382-a212-1ab1d36d6eab None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:19.083151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 458/889] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:19.088719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:19.088955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:19.089234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:19.107699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:19.108919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:19.126809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:19.191505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 432/890] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:19.214948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb99144-a188-4382-a212-1ab1d36d6eab None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:19.332748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb99144-a188-4382-a212-1ab1d36d6eab None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:19.360469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:19.369437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:19.370251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:19.370534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:19.370870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:19.382115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:19.390405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:19.392622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:19.394369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 459/891] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 11:51:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:19.398291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:19.506033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe9f69-1fdf-4f09-ae3e-227bb00990f9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:19.541185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:19.541859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:19.542149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:19.542425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:19.543226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:19.543534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:19.544920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:19.556114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 433/892] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:51:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:19.565025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:19.673017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5111370-0a6f-4146-98dc-ecf9ba2cd08f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:19.707992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:19.708794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:19.709088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:19.709370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:19.720632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:19.722172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:19.740438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:19.742566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:19.744715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:19.744715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:19.744715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:19.744715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:19.744715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 460/893] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:19.747825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:19.766147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:19.855432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753d408-c4cd-4b41-ac42-ec627cffeaf9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:19.892059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:19.892731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:19.893022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:19.893306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:19.894711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:19.895057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:19.896441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:19.908478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 434/894] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:19.913802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:19.959786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.024648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc571f9-50de-4585-9fa3-f24b21a242aa None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:20.060460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:20.061271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.061551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.061824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.070080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:20.071591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:20.087329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:20.089505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:20.091263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 461/895] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:51:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:20.096449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:20.176376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.215441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0915e-063d-42cf-94ea-710afcac8aba None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:20.251193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:20.251890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.252172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.252459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.253432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:20.253746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:20.255276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c71a-502a-4b65-9a34-8fa787fcb8c0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:20.276690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 435/896] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 11:51:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:20.284329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:20.392684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.396996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bd1f6-4ada-4f5b-be59-214b3651db2a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:20.431634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:20.432280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.432562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.432838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.502398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:20.504334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:20.515711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868785e-7d20-404d-9e95-11e98792613e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.516849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868785e-7d20-404d-9e95-11e98792613e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.516849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868785e-7d20-404d-9e95-11e98792613e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.519744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 436/897] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:20.525955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c2ac8-c1cd-4251-b042-3506d3fcc8f1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.525955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c2ac8-c1cd-4251-b042-3506d3fcc8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.525955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c2ac8-c1cd-4251-b042-3506d3fcc8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.541415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ef3c2ac8-c1cd-4251-b042-3506d3fcc8f1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:20.542899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ef3c2ac8-c1cd-4251-b042-3506d3fcc8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:20.609774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.627060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 437/898] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:20.627263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 462/899] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:20 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:20.627352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84745-93bc-4bee-b776-b3610878f5a0 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.627352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84745-93bc-4bee-b776-b3610878f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.627491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:20.627574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c84745-93bc-4bee-b776-b3610878f5a0 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.627574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 438/900] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:20.628610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.628899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.629186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.647774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:20.648967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:20.713864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.727628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 439/901] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:20.728834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:20.734891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afaf5-43c3-4c49-9e0b-d1cafdf58f8c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:20.775127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:20.775997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.777060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.777349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.789889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:20.791827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:20.814201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:20.815147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:20.816672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:20.816672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:20.816672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:20.816672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:20.817598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 463/902] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:20.818486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:20.847522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790e4a-2772-4590-b1ec-d6e520088f85 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:20.872167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.885592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:20.886352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.886789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.886966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.887587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:20.888961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:20.901483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:20.902493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:20.903375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 440/903] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:20.904181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:20.936745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59628b95-cfa3-4170-bc2e-329ad7c240d2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:20.963410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:20.974646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:20.975485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:20.975843 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:20.976221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:20.977010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:20.978508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:20.993382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 464/904] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:20 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:21.000106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.024275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665f37d-fbe2-448d-a5f8-f4decf4683d0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.088013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.089366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.089906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.090401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.091912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:21.092440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.095051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.119102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 441/905] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:21.120927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.139622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f64039-89fb-490c-91fc-beb3b3228dde None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.153255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:21.185051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.185721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.186006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.186284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.187145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:21.187454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.189314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.195927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 465/906] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:21.201399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.261018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a968d-15b5-4180-b8ad-4dbdf314ab8a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.276223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:21.309574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.310976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.311770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.312533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.318187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65eb80-daec-4e94-961c-f4cb94603e9c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.323312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.325001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8fc23f6e-dbcf-4b06-b4f0-38e801a091a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:21.344344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:21.345296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:21.346209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:21.353912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 442/907] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:21.355508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3769-d468-46d4-8ee2-60530c1b4705 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.355810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3769-d468-46d4-8ee2-60530c1b4705 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.356097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3769-d468-46d4-8ee2-60530c1b4705 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.357413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 443/908] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:21.358111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3769-d468-46d4-8ee2-60530c1b4705 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.364123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.365009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.365462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.365890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.366751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:21.367260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.369279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d4cab-e205-4dea-9630-23e5227f5f71 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.375990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 466/909] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:21.378929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.379374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.379659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.397413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:21.398615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:21.429719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:21.470565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 467/910] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:21.475065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3769-d468-46d4-8ee2-60530c1b4705 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.479854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.511024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.511785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.512076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.512420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.513208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:21.513518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.515196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.533008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 444/911] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:21.533798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.569409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:21.593653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25320-4d03-4e70-b3b7-d75720f625f7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.630599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.631198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.631483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.631757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.632496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.634083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.642480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a140f6-5a63-4ed9-bef1-58cf78a3f2e5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.653846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 468/912] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:21.654742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.699333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:21.707989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.709166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.709666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.710068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.710908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.713816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:21.731366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:21.732455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:21.733614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 445/913] 10.222.0.41 () {56 vars in 1132 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:21.740008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:21.808386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb932a-b2ce-47da-bb76-e549d63509ed None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.848606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.849692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.850217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.850703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.851867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] RBAC: Adding query filter params (role.id=3467f6f457b949a7a92d0e944b1acec3, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=5b62f565650b42f5b4a1bc2f8b39b439) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:21.852400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.854118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.861763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 469/914] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/role_assignments?user.id=5b62f565650b42f5b4a1bc2f8b39b439&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=3467f6f457b949a7a92d0e944b1acec3 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:21.872432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:21.886285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2c9b-02f1-45dc-88df-27e7564ce916 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:21.929766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:21.930741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:21.931239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:21.931689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:21.932636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:21.933164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:21.934799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:21.941642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 446/915] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:21.947171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:22.055513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:22.069798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8fd14-a862-47ca-b45e-dd439cca638c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:22.104894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:22.105562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:22.105851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:22.106131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:22.106857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:22.107195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:22.108586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:22.113853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 470/916] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:22.119673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:22.224889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33410fa-4359-4484-a9cc-8125e4387a67 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:22.261846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:22.262575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:22.262889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:22.263176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:22.263976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] RBAC: Adding query filter params (interface=public, service_id=be8b4a2ea5d642b4a7f388a7fc5b2fd8, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:22.264352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:22.265794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8da691-9f16-4afa-87e3-955c0f1aaaad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:22.272197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 447/917] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:22 2022] GET /identity/v3/endpoints?service_id=be8b4a2ea5d642b4a7f388a7fc5b2fd8&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:22.277743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:22.383467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9eab6-03b8-42af-b070-c2cb915ba55b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:22.419584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371aa9d-0ca4-4119-be62-99a05787d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:22.420595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371aa9d-0ca4-4119-be62-99a05787d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:22.421101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371aa9d-0ca4-4119-be62-99a05787d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:22.421583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371aa9d-0ca4-4119-be62-99a05787d1e9 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:22.422491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371aa9d-0ca4-4119-be62-99a05787d1e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:22.424113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371aa9d-0ca4-4119-be62-99a05787d1e9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:22.431639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 471/918] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:22 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 11:51:23.387676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38160a-10a1-496e-814f-9f6770820a21 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:23.388203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38160a-10a1-496e-814f-9f6770820a21 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:23.388203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38160a-10a1-496e-814f-9f6770820a21 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:23.389840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 448/919] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:23.395190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:23.395597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:23.395767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:23.414023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:23.415260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:23.426624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:23.486619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 472/920] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:23.509922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38160a-10a1-496e-814f-9f6770820a21 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:23.535982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:23.626959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38160a-10a1-496e-814f-9f6770820a21 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:23.664033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:23.665204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:23.665715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:23.666195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:23.667096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:23.668672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:23.681542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:23.682580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:23.683455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:23.686295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 449/921] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:23.692499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:23.799292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cdd34-dbc8-445c-9261-d97c82d312f7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:23.840022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:23.840711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:23.840999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:23.841533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:23.842259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:23.842568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:23.844043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:23.850817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 473/922] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:23.856432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:23.950171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:23.970874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872e08-8caf-48b4-83d9-a9cc05d237ea None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.007512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.008574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.009070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.009553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.010474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:24.011537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.013150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a27f9-7fde-4a4a-8fb8-89910a7b6125 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.019646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 450/923] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:24.026162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.039293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.133939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cf0d7-0da3-4d44-9d84-ac8f3e6b4e71 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.156161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccc33b-d343-477b-bf5c-65b6aa8d29cc None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.156352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccc33b-d343-477b-bf5c-65b6aa8d29cc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.156715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccc33b-d343-477b-bf5c-65b6aa8d29cc None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.158295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 451/924] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:24.164187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5948b39-0f19-46e5-9fb3-deec8c053fe8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.164347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5948b39-0f19-46e5-9fb3-deec8c053fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.164694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5948b39-0f19-46e5-9fb3-deec8c053fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.174940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.175906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.176412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.177029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.177825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] RBAC: Adding query filter params (interface=public, service_id=3f8ded31aee4406ab6541b90de821832, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:24.178294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.179962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.183352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c5948b39-0f19-46e5-9fb3-deec8c053fe8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:24.184781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c5948b39-0f19-46e5-9fb3-deec8c053fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:24.185752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 474/925] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/endpoints?service_id=3f8ded31aee4406ab6541b90de821832&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:24.192222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.266945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 452/926] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:24.266945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879cefa-db9f-4826-93e7-9165834ddae7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.266945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879cefa-db9f-4826-93e7-9165834ddae7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.266945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879cefa-db9f-4826-93e7-9165834ddae7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.266945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 453/927] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:24.271205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.272041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.272041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.272341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.292366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:24.293796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:24.303284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82569a16-8339-4d83-8ee4-692f589d5431 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.342592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.343277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.343555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.343834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.344589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.345887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.362879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 475/928] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 11:51:24.362879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.387317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 454/929] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:24.399806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.410286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.497508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67cae9-ceae-4a94-be8c-327afa3f29b4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.533483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a19fa-2212-42f2-b38a-ff355a76e3a3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.537840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.538904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.539405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.539884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.540804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.542676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.544458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING keystone.server.flask.common [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 06 11:51:24.563429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.566317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.sql.core [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 11:51:24.566317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 11:51:24.566317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [parameters: {'id': 'caddfbe0af42412b876136a5d95bbfe7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 11:51:24.566317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80147) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 11:51:24.584040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.584866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.585152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.585426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id caddfbe0af42412b876136a5d95bbfe7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id caddfbe0af42412b876136a5d95bbfe7. Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application conn.query(q) Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 11:51:24.588308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result.read() Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 06 11:51:24.589124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application next(self.gen) Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application yield resource Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application next(self.gen) Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise self.value Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application session.commit() Jun 06 11:51:24.590070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.session.flush() Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application compat.raise_( Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 11:51:24.591519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 11:51:24.593500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application conn.query(q) Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application result.read() Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 11:51:24.594483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [parameters: {'id': 'caddfbe0af42412b876136a5d95bbfe7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.595712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:24.597669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id caddfbe0af42412b876136a5d95bbfe7. Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.604967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 476/930] 10.222.0.41 () {60 vars in 1140 bytes} [Mon Jun 6 11:51:24 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:24.606437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:24.606597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:24.608333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:24.610530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:24.613508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 455/931] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:24.618403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.702592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.742745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965fe24-95c5-4eea-b972-ac607f9c4ba3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.759238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae371-51df-40d4-adea-4942a534f03c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.780997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.781673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.781959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.782242 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.783022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:24.783360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.785462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ad5cf-3681-4c14-9be8-7128e4b2f706 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.794256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 456/932] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:24.795670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4de0-9a0e-4e21-9bc4-89730ae99582 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.795967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4de0-9a0e-4e21-9bc4-89730ae99582 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.796256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe4de0-9a0e-4e21-9bc4-89730ae99582 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.798879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 457/933] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:24.799459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe4de0-9a0e-4e21-9bc4-89730ae99582 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.801203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.802403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.802701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.803005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.811803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.814269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a9528af9-01f3-4952-9521-9f06a60e6e56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:24.831673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:24.832675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:24.833664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 477/934] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:24.834863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.834863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.835008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.854899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:24.856865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:24.911659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.916150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe4de0-9a0e-4e21-9bc4-89730ae99582 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:24.941947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 478/935] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:24.943016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:24.945837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:24.955326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:24.956725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:24.957021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:24.957308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:24.968844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:24.970327 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:24.996705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:24.997891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:24.999016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:24.999016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:24.999016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:24.999016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:25.004835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 458/936] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:25.006214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.074405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477c0ad-7bec-4fe3-a13a-94da314feb23 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.114091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.114758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.115079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.115363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.116115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:25.116429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.117872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.118943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc3627-c1ef-4e3d-bd7c-6d29c764b8ce None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.139220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 479/937] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:25.140065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.169837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.171077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.171594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.172178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.173156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.174941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:25.189199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:25.191309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 11:51:25.192273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:25.195262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 459/938] 10.222.0.41 () {56 vars in 1132 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:25.201440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.203627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:25.261611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:25.275744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86349eb4-55b9-41cf-b371-db488c6cc300 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.323076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.323868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.324199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.324620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.325463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:25.325885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.327552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.342809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 480/939] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:25.351545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.362586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845ef5d-6fcb-4f84-ab56-da2e47611f0f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.414630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.415321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.415602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.415887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.416597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:25.416913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.418292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.425443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 460/940] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:25.431936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.495651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3839-b98f-491a-8b8b-ff264d4db3c6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.498887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:25.539572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.541322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.542016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.542512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.554243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.556714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.558125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263a01-7408-42ad-a8e3-1540f3151e33 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.568952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:25.578094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:25.579242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:25.580405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:25.583196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 481/941] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:25.589249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.606928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.608502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.608999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.609467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.610524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:25.611406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.613273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.620599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 461/942] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:25.626740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.722515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be2ff-58c3-42ea-a0e6-ef4fa88ccc45 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.745433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73bfe1-c04b-4a91-a3f2-6e76e7ad4552 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.758931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.760012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.760486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.760970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.762025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:25.762546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.764262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.788873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.788873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.788873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.788873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.789297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.789297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:25.789668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 482/943] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:25.791054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:25.818925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 462/944] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:51:25.843327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:25.920338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cee3b-f56b-4112-a4d9-f47294e74c84 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:25.970760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:25.972021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:25.972442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:25.972795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:25.984371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:25.986002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:26.006008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:26.007035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:26.011663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:26.011663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:26.011663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:26.011663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:26.011663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 483/945] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:26.015534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:26.052035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:26.132758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c85a58-fb1a-448e-a732-65c02c3114f8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:26.169208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:26.170151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:26.170623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:26.171075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:26.172009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:26.172479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:26.173998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:26.186591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 463/946] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:26.192529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:26.265937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:26.298276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d20ad-d7d7-48dd-8393-f71d7c47e6d2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:26.337717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:26.338503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:26.338786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:26.339095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:26.346938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:26.348526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:26.362551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:26.363518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:26.364410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 484/947] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:26.371081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:26.469216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:26.482816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185218d-7b23-4bc5-9c1c-bd99d34f06e9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:26.522712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:26.523437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:26.523724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:26.524002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:26.524801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:26.525111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:26.526573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dddb1-d542-4d16-bb19-ee56428194e6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:26.540354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 464/948] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:26.546226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:26.660129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b4576-2973-4040-98a1-3ec505794e94 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:26.689823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:26.696401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd575-c045-4e27-bd44-140ccf0459b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:26.697383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd575-c045-4e27-bd44-140ccf0459b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:26.697667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd575-c045-4e27-bd44-140ccf0459b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:26.697971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd575-c045-4e27-bd44-140ccf0459b6 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/5b62f565650b42f5b4a1bc2f8b39b439/roles/3467f6f457b949a7a92d0e944b1acec3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:26.700612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd575-c045-4e27-bd44-140ccf0459b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=5b62f565650b42f5b4a1bc2f8b39b439, role_id=3467f6f457b949a7a92d0e944b1acec3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:26.702086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd575-c045-4e27-bd44-140ccf0459b6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:26.719377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 485/949] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:26 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/5b62f565650b42f5b4a1bc2f8b39b439/roles/3467f6f457b949a7a92d0e944b1acec3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 11:51:26.866915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b2c15-960c-4405-a4e1-014ba204697c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:26.867249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b2c15-960c-4405-a4e1-014ba204697c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:26.867488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b2c15-960c-4405-a4e1-014ba204697c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:26.869153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 465/950] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:26.874889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:26.875195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:26.875486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:26.894439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:26.895752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:26.969938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 486/951] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:26.998631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b2c15-960c-4405-a4e1-014ba204697c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:27.115154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b2c15-960c-4405-a4e1-014ba204697c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:27.152648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:27.153460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.153742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.154015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.154660 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:27.156053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:27.168925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:27.169987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.170885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 466/952] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:27.179879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:27.293274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad14e5-44c7-4c6d-9bc5-19c44a2987b8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:27.317304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:27.337240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:27.337924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.338217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.338503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.339250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:27.339567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:27.340975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:27.347615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 487/953] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:27.356218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:27.373558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:27.476595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d25d-dcb2-4336-8474-9be8baa239bd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:27.519665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:27.520341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.520627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.520907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.521608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:27.521915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:27.523339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bfcdf-8c14-43b1-bb97-0610b61bd698 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:27.528710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 467/954] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:27.534440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:27.601958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34cbb5-b865-4011-9f73-dbfb49a32ab2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.602311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34cbb5-b865-4011-9f73-dbfb49a32ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.602526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34cbb5-b865-4011-9f73-dbfb49a32ab2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.606910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 468/955] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:27.609172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.609465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.609749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.628955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:27.630145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:27.643830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea241ff8-438d-4abb-a581-c3284988392f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:27.682264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:27.683076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.683364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.683642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.684438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:27.685929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:27.706954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 469/956] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:27.708926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 488/957] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:27.726140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:27.734986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:27.850966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bb253-2151-4049-afdb-f62cd028aed5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:27.899983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:27.900580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:27.900873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:27.901153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:27.901870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:27.903473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:27.904809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING keystone.server.flask.common [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 06 11:51:27.922022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.sql.core [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 11:51:27.922022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 11:51:27.922022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [parameters: {'user_id': 'f518445825584aabbad747d94d642c0c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 11:51:27.922022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 06 11:51:27.929679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 11:51:27.930600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application yield resource Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application next(self.gen) Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise self.value Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application session.commit() Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 11:51:27.931634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.session.flush() Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application compat.raise_( Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 11:51:27.932563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 11:51:27.933492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application conn.query(q) Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application result.read() Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 11:51:27.934391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f518445825584aabbad747d94d642c0c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:27.939867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 06 11:51:27.942115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:27.944204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 470/958] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 11:51:27 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:27.945030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:27.995363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:28.073396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88157d42-3cde-4b7c-8efa-92e9da653bd3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:28.113814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:28.114680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:28.114995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:28.115276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:28.123672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:28.125128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f2c98722-5725-4394-b73c-0418c7beb120 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:28.144934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:28.145973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 11:51:28.146970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:28.149758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 489/959] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:28.155819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:28.226485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:28.266865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d48e6-8be4-4967-8012-342eac0ccc64 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:28.302367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:28.303076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:28.303413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:28.303698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:28.304501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:28.304808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:28.306442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cbd6b-72ad-47c2-aa2b-81a2e2f63d5d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:28.350984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 471/960] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 11:51:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:28.427170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae27643-e61f-4be6-abe3-d967228acf63 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:28.427355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae27643-e61f-4be6-abe3-d967228acf63 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:28.427650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae27643-e61f-4be6-abe3-d967228acf63 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:28.429288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 490/961] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:28.435312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:28.435609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:28.435896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:28.454064 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:28.455666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:28.535584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 472/962] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:28.558375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae27643-e61f-4be6-abe3-d967228acf63 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:28.673124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:28.678535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae27643-e61f-4be6-abe3-d967228acf63 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:28.714365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:28.715186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:28.715473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:28.715753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] PATH_INFO: `/v3/services/compute` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:28.716658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:28.718033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:28.730697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:28.732208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:28.733060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 491/963] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:28.739260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:28.793331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:28.842512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bb26a-eb0e-4f53-ae98-385560e17c20 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:28.877389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:28.878365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:28.878703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:28.879024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:28.879771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:28.880084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:28.881455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bf3bb-e8db-4ef6-9d8d-0fd99efbb42b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:28.890724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 473/964] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:28.892095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:29.006756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a226f2f-e5a2-462c-b57f-72c9eb252ced None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:29.043483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58322010-9a55-4965-a317-97300a640347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:29.044239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322010-9a55-4965-a317-97300a640347 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:29.044596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322010-9a55-4965-a317-97300a640347 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:29.044945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322010-9a55-4965-a317-97300a640347 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:29.045722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58322010-9a55-4965-a317-97300a640347 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:29.046104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58322010-9a55-4965-a317-97300a640347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:29.047595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58322010-9a55-4965-a317-97300a640347 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:29.058889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 492/965] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:29.262075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183bdac-b271-4177-bc69-f12d06dd63bb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:29.262301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183bdac-b271-4177-bc69-f12d06dd63bb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:29.262588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183bdac-b271-4177-bc69-f12d06dd63bb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:29.264148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 474/966] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:29.269270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:29.269569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:29.269859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:29.286793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:29.288017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:29.312274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:29.360550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 493/967] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:29.384422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183bdac-b271-4177-bc69-f12d06dd63bb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:29.489154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183bdac-b271-4177-bc69-f12d06dd63bb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:29.506278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:29.527542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:29.528359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:29.528724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:29.529221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:29.543478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:29.557614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:29.558540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:29.559752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:29.562929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 475/968] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 11:51:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:29.566281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:29.669744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1767-2366-4344-a66c-261705c05b24 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:29.704183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:29.704928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:29.705143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:29.705428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:29.706742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:29.707261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:29.708652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:29.717243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 494/969] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:51:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:29.722817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:29.833777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94f77-760e-4501-b7a0-17fa988988f6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:29.867647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:29.868444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:29.868731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:29.869009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:29.879668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:29.881125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:29.898520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:29.899615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:29.905155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:29.905155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:29.905155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:29.905155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:29.905155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 476/970] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:29.906828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:29.931339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.021840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6f77d-de83-4d0b-8663-72a3e3a938e1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.056475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.057257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.057639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.058014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.058863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:30.059276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.060734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f150109-b5ae-42b6-acef-cdf9ef994750 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.073304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 495/971] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:30.078643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.109169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.114499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf827-ac26-45b2-ad33-999795136fee None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.114800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf827-ac26-45b2-ad33-999795136fee None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.115116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf827-ac26-45b2-ad33-999795136fee None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.116765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 496/972] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:30.121733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.122015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.122298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.139205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:30.140402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:30.185173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f80a4-1248-4b10-8e4e-773e76dbeaeb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.213622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 497/973] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:30.221771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.222695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.223099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.223449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.232347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.233969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.237275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.249013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:30.249700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:30.250638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:30.251642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 477/974] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:30.257187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.349682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0584141-d41c-45cc-b9f5-8c479a89885f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.352197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.366744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a21a5-af59-4c24-8791-eccae6c71a5b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.389059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.390106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.390391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.390671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.391352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.392736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.405138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.405809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.406099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.406381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:30.406704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:30.407818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:30.412045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:30.412797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:30.412797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.412940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 498/975] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:30.413011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.414516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.429756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 478/976] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:30.435289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.504041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.538028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b483f0-d801-4f03-ad49-95f8b6c980ad None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.539033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aecbee-a2a6-4dcc-88b4-eae76a9a5bbc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.573071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.573724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.574010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.574303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.575002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:30.575313 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.575472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.576119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.576406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.576679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.576776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.582263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 499/977] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:30.587559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.594078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.653193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.655730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.697573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4b82c-b0de-4d74-93b8-3d5de8f13456 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.733799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.734570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.734875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.735164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.735877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:30.736186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.738262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.746868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 500/978] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:30.748939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.752544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.774885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 479/979] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 11:51:30.780306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.829900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:30.857217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2052d2-d3a0-489d-b39e-237527462ee4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.889775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68820b5d-99c2-415d-b34b-b149833e5d7e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:30.891729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.892401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.892689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.892982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.894538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] RBAC: Adding query filter params (interface=admin, service_id=3f8ded31aee4406ab6541b90de821832, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:30.894878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.896287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.906571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 501/980] 10.222.0.41 () {56 vars in 1249 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/endpoints?service_id=3f8ded31aee4406ab6541b90de821832&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:30.907276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.927540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:30.928338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:30.928624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:30.928901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:30.939839 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:30.942005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:30.961092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:30.962053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:30.963001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:30.963001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:30.963001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:30.963001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:30.963001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 480/981] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:30.967873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:30.980070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:31.025627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae311e-4720-4da3-a44a-d0d07282d99d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:31.063767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:31.064441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.064725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.065585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.066431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:31.068048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:31.074147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 502/982] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 11:51:31.092396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:31.097962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43aa26-3d45-4397-9616-d10a875a1cb3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:31.137159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:31.137853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.138139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.138419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.139200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:31.139511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:31.140938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:31.160384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 481/983] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:31.161926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:31.271896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f0827-7f08-4ca3-8ac6-1bab02e7a5fc None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:31.306518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:31.311753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:31.312563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.312848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.313128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.320815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:31.322286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-96c38f86-7e50-4bd7-8f2a-b0a9a08a8282 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:31.336672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:31.337581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:31.338484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 503/984] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:31.345815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:31.377971 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb902e-25c3-487c-a260-a333c096aa90 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.378334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb902e-25c3-487c-a260-a333c096aa90 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.378519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb902e-25c3-487c-a260-a333c096aa90 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.380066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 504/985] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:31.385770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.386075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.386360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.405307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:31.406524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:31.455200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1f0d9-ad28-4584-be5f-90f1db56b584 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:31.479658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 505/986] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:31.491812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:31.492480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.492768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.493046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.493827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:31.494139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:31.495604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:31.503113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:31.509960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 482/987] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:31.515657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:31.537479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:31.615968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970d30-db18-4ff0-8b6a-b1f0fe233279 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:31.628821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915fff-7d21-4a2a-a5d1-e1e41d28412e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:31.638372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:31.654478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5169a29-f54f-4b78-aa09-994a8601445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:31.655179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169a29-f54f-4b78-aa09-994a8601445c None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.655471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169a29-f54f-4b78-aa09-994a8601445c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.656040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5169a29-f54f-4b78-aa09-994a8601445c None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.656471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169a29-f54f-4b78-aa09-994a8601445c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:31.657873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5169a29-f54f-4b78-aa09-994a8601445c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:31.675546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 506/988] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:31 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:51:31.693359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:31.694052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:31.694340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:31.694634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:31.695542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] RBAC: Adding query filter params (role.id=3467f6f457b949a7a92d0e944b1acec3, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=5b62f565650b42f5b4a1bc2f8b39b439) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:31.695875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:31.697357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536e77f-30f3-4e32-8566-4ce71e5d6c5c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:31.704617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 483/989] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:31 2022] GET /identity/v3/role_assignments?user.id=5b62f565650b42f5b4a1bc2f8b39b439&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=3467f6f457b949a7a92d0e944b1acec3 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:33.343234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a3d5a-2a42-46c3-8b9d-24a3d0fe6c00 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.343756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a3d5a-2a42-46c3-8b9d-24a3d0fe6c00 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.343835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a3d5a-2a42-46c3-8b9d-24a3d0fe6c00 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:33.345291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 507/990] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:33.350927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.351348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.351520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:33.368230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:33.369399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:33.438859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 484/991] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:33.462147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a3d5a-2a42-46c3-8b9d-24a3d0fe6c00 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:33.568524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a3d5a-2a42-46c3-8b9d-24a3d0fe6c00 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:33.602582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:33.603346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.603631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.603904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:33.604544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:33.605932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:33.618343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:33.619382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 11:51:33.620233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:33.621913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 508/992] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 11:51:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:33.625565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:33.636376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:33.740012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d92443-e6ee-4d78-a469-cff9f67a2a18 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:33.777735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:33.778403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.778691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.779015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:33.779722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:33.780034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:33.781404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:33.791556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 485/993] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:33.793061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:33.840255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:33.898132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0353a-dfd5-45d2-bbc7-06e881a2c1e4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:33.939327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:33.940526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.940816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.941192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:33.941912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:33.942229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:33.943652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24305402-4696-4eaf-9cf2-6c44c875657c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:33.948855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 509/994] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 11:51:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:33.954600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:33.989589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77df72-add2-4d3f-a469-6462e3b46d1c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.989883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77df72-add2-4d3f-a469-6462e3b46d1c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.990166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77df72-add2-4d3f-a469-6462e3b46d1c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:33.992154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 510/995] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:33.996444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4309c-2776-4b89-873b-b130630238b3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:33.996738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4309c-2776-4b89-873b-b130630238b3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:33.997023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4309c-2776-4b89-873b-b130630238b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.018410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7ff4309c-2776-4b89-873b-b130630238b3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:34.019654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7ff4309c-2776-4b89-873b-b130630238b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:34.035022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.064009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fdec1-9406-4c67-936e-e35f66cc84c6 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.093622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 511/996] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:34.095101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca662fce-ba8a-4025-9c47-1503c68a3f55 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.095392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca662fce-ba8a-4025-9c47-1503c68a3f55 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.095672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca662fce-ba8a-4025-9c47-1503c68a3f55 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.097623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 512/997] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:34.102736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.103063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.103345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.109792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.110632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.110958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.112614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.113388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.114771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.122950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:34.124193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:34.129599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 486/998] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:34.132713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.157069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.224920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 513/999] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:34.247772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.254915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb091f-5535-4f88-9642-bbaf7d9cbc95 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.290121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.299502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.301490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.302415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.303208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.304316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.306268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:34.320786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:34.321952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:34.323132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 487/1000] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:34.328996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.365768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc097b-efb9-472d-ad69-4b56febd4c76 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.406746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.407933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.408181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.409942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] PATH_INFO: `/v3/services/image` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.410629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.412077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:34.428118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:34.429173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:34.435332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 514/1001] 10.222.0.41 () {56 vars in 1105 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:34.437898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.440314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a64da7-7513-4c86-b782-06279bb678ea None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.472505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.477442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.478327 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.478685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.479074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.479856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:34.480236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.481667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.487400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 488/1002] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:34.492997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.555130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.558292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d40c13-f4ce-43b3-b081-199c9eafd53f None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.594768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.595463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.595750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.596787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.597492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] RBAC: Adding query filter params (name=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:34.597807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.599217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.599890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cdf20-40dc-4d9b-913f-29a82b94a75a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.609097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 515/1003] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:34.610117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.639789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.640480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.640767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.641053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.641754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:34.642066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.643962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.653649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 489/1004] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:34.654765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.712682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.724000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b762b-09b6-4816-b371-52e3ddd61229 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.765392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.766115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.766401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.766686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.767436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] RBAC: Adding query filter params (type=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:34.767959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.770249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.774793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03e911-58a0-419c-88eb-a13322ec8437 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:34.776966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 516/1005] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:34.814113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:34.827889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:34.828557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:34.828841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:34.829119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:34.829910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] RBAC: Adding query filter params (interface=public, service_id=cce90735a7604d6ab1f8456f38a4ecce, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:34.830223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:34.832823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ed763-9995-4200-8dd7-37c56db44876 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:34.839116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 490/1006] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/endpoints?service_id=cce90735a7604d6ab1f8456f38a4ecce&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:34.844824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:34.959897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ce1f0-614e-4ed8-b0ff-e9fb37bcb919 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:35.000608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae46d5-d0e2-43f1-a2a8-1784bb9acdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:35.004782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae46d5-d0e2-43f1-a2a8-1784bb9acdbf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:35.004782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae46d5-d0e2-43f1-a2a8-1784bb9acdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:35.004782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae46d5-d0e2-43f1-a2a8-1784bb9acdbf None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:35.004782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae46d5-d0e2-43f1-a2a8-1784bb9acdbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:35.004782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae46d5-d0e2-43f1-a2a8-1784bb9acdbf None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:35.011799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 517/1007] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:34 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 11:51:36.434630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af53c-829b-4d07-b6b2-9e21114d3c48 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:36.435241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af53c-829b-4d07-b6b2-9e21114d3c48 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:36.435241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af53c-829b-4d07-b6b2-9e21114d3c48 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:36.436737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 491/1008] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:36.442246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:36.442643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:36.442811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:36.460332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:36.461512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:36.476867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:36.530589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 518/1009] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:36.554602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626af53c-829b-4d07-b6b2-9e21114d3c48 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:36.659383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626af53c-829b-4d07-b6b2-9e21114d3c48 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:36.693548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a3bd4-e17f-47d8-9df4-b9dbe5569550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:36.694259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3bd4-e17f-47d8-9df4-b9dbe5569550 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:36.694546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3bd4-e17f-47d8-9df4-b9dbe5569550 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:36.694828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3bd4-e17f-47d8-9df4-b9dbe5569550 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:36.696033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a3bd4-e17f-47d8-9df4-b9dbe5569550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:36.697469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a3bd4-e17f-47d8-9df4-b9dbe5569550 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:36.703735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 492/1010] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:36.705044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:36.811997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b2f9-8d04-4f28-a503-fe7b993d53cb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:36.847358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f27b0e-c93a-4e9c-876f-5255a773f211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:36.848121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f27b0e-c93a-4e9c-876f-5255a773f211 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:36.848483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f27b0e-c93a-4e9c-876f-5255a773f211 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:36.848838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f27b0e-c93a-4e9c-876f-5255a773f211 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:36.849623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f27b0e-c93a-4e9c-876f-5255a773f211 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:36.851157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f27b0e-c93a-4e9c-876f-5255a773f211 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:36.878789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 519/1011] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:36 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:51:37.135052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a51bb9-4041-4f3c-938e-cf14e950ee96 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.135405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a51bb9-4041-4f3c-938e-cf14e950ee96 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.135615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a51bb9-4041-4f3c-938e-cf14e950ee96 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.136999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 493/1012] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:37.142362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507b5a-ce8a-49ea-a99d-2a62d050797c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.142658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507b5a-ce8a-49ea-a99d-2a62d050797c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.143612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507b5a-ce8a-49ea-a99d-2a62d050797c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.149362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:37.164735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b2507b5a-ce8a-49ea-a99d-2a62d050797c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:37.165917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b2507b5a-ce8a-49ea-a99d-2a62d050797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:37.235694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 520/1013] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:37.260284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a51bb9-4041-4f3c-938e-cf14e950ee96 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:37.321052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:37.325422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf17bf-ffbd-4912-a8b1-ddd34cd142cc None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.325716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf17bf-ffbd-4912-a8b1-ddd34cd142cc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.326003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf17bf-ffbd-4912-a8b1-ddd34cd142cc None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.327416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 521/1014] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:37.333093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.333380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.333664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.351160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:37.352354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:37.378227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a51bb9-4041-4f3c-938e-cf14e950ee96 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:37.417578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:37.418254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.418543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.418823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.419579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:37.420949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:37.428688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 522/1015] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:37.435329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 494/1016] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:51:37 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:37.452388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:37.524109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:37.567270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde2dce-0baa-4bcb-97f6-9d922b6a2985 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:37.610964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:37.612268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.612556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.612829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.613694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:37.615133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:37.627508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:37.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:37.629468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 523/1017] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:37.636974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:37.747107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138152-8023-43c8-aaf7-43989f6a214b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:37.763866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:37.783971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:37.784693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.785047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.785329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.786029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:37.786337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:37.787726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:37.792844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 495/1018] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:37.798595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:37.900541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebead01c-34b8-4b70-a1d9-0f3b4519407c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:37.922993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:37.936395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:37.937187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:37.937594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:37.937976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:37.938783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:37.939218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:37.940890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a22f03-449c-4399-808b-12eaa2d6031d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:37.950530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 524/1019] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 11:51:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:37.951971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:38.061448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c065bc-b121-44fe-b6f3-aed0418e9c8e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:38.097911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ce52-e90c-43fa-9225-a6b813d482ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:38.098577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ce52-e90c-43fa-9225-a6b813d482ee None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:38.098886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ce52-e90c-43fa-9225-a6b813d482ee None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:38.099175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ce52-e90c-43fa-9225-a6b813d482ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:38.099913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ce52-e90c-43fa-9225-a6b813d482ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:38.101284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ce52-e90c-43fa-9225-a6b813d482ee None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:38.122357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 496/1020] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:51:39.193006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8ea9e-62f5-4cdf-9bad-739b26712db0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:39.193544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8ea9e-62f5-4cdf-9bad-739b26712db0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:39.193632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8ea9e-62f5-4cdf-9bad-739b26712db0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:39.195072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 525/1021] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:39.200300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:39.200715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:39.200880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:39.218183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:39.219378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:39.250096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:39.297586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 497/1022] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:39.322540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8ea9e-62f5-4cdf-9bad-739b26712db0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:39.428185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8ea9e-62f5-4cdf-9bad-739b26712db0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:39.464517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ad1d-19c0-475d-91b4-0f39a072685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:39.465311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ad1d-19c0-475d-91b4-0f39a072685c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:39.465600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ad1d-19c0-475d-91b4-0f39a072685c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:39.465878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ad1d-19c0-475d-91b4-0f39a072685c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:39.467355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22ad1d-19c0-475d-91b4-0f39a072685c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:39.468837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22ad1d-19c0-475d-91b4-0f39a072685c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:39.470684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 526/1023] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:39.476351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:39.583062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300b8c-2d5a-47dc-bc68-6df796c484ef None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:39.617718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2816e-1bdd-4802-8512-f90aaaa84ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:39.618386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2816e-1bdd-4802-8512-f90aaaa84ea6 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:39.618670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2816e-1bdd-4802-8512-f90aaaa84ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:39.618971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2816e-1bdd-4802-8512-f90aaaa84ea6 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:39.619673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2816e-1bdd-4802-8512-f90aaaa84ea6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:39.621093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2816e-1bdd-4802-8512-f90aaaa84ea6 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:39.642559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 498/1024] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:51:39 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:39.769007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3aae9-1333-4d95-9de7-645526848544 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:39.769365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3aae9-1333-4d95-9de7-645526848544 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:39.769587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3aae9-1333-4d95-9de7-645526848544 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:39.771193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 527/1025] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:39.776853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:39.777154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:39.777441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:39.795192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:39.796411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:39.812228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:39.875096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 499/1026] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:39.898287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3aae9-1333-4d95-9de7-645526848544 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:39.958358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:40.006794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3aae9-1333-4d95-9de7-645526848544 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.042689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.043506 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.043790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.044065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.044712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:40.046082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:40.057846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:40.059256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:40.065844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 528/1027] 10.222.0.41 () {56 vars in 1105 bytes} [Mon Jun 6 11:51:39 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:40.068136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:40.173319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28efcdb-4e72-47aa-9fb2-7b2b7c36edba None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.209103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.209751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.210143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.210313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.211037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] RBAC: Adding query filter params (name=image) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:40.211346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:40.212712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:40.222790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 500/1028] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:40.224413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:40.326066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d96e9-c6f7-4a87-9261-98e47a89e5f4 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.344137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:40.362562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.363254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.363545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.363828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.364533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] RBAC: Adding query filter params (type=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:40.364844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:40.366223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40fe9c-a287-48eb-bc72-a482159f6a17 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:40.375849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 529/1029] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:40.377903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:40.392790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:40.462720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91398fb-a75a-4f04-8065-309d5f897dc7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.463023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91398fb-a75a-4f04-8065-309d5f897dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.463391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91398fb-a75a-4f04-8065-309d5f897dc7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.465104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 530/1030] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:40.470665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.471109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.471529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.481499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52116fe-6358-4685-a0af-c4cbb6802b86 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.488534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:40.489725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:40.518368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.519423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.519714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.520001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.520794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] RBAC: Adding query filter params (interface=public, service_id=24a5272b38db4c71bdb6afa6fe69b9f6, region_id=RegionOne) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:40.521114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:40.522491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:40.527735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 501/1031] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/endpoints?service_id=24a5272b38db4c71bdb6afa6fe69b9f6&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:40.534878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:40.571976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 531/1032] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:40.596045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:40.622570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:40.648902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d26a19-3ec0-4f3d-a4e9-a0ab894f948c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.683953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.684620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.684909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.685191 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.685860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:40.687256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:40.694407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 502/1033] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 11:51:40.702832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00189fe8-fd50-44ff-bdd4-fd4b62465d7d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.719167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:40.746005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.746864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.747148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.747429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.764001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:40.773582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:40.774627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:40.775846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:40.776491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 532/1034] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:40.781784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:40.896853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbc62e-5cc3-49b8-9ec9-5b7d2d853ea4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:40.934863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:40.935572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:40.935862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:40.936145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:40.936889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:40.937202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:40.938575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:40.948194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 503/1035] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:40.953514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:41.066044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcc051-0844-4bf8-9ecf-800485bc5bda None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:41.101671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:41.102464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:41.102748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:41.103054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:41.114361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:41.115974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:41.133837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:41.134745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:41.135724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:41.135724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:41.135724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:41.135724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:41.135724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 533/1036] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:41.141682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:41.144902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:41.244560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2d1d4-f5b7-4c1d-8ecb-54d9afbf2695 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:41.279404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:41.279991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:41.280277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:41.280554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:41.281269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:41.281576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:41.282964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:41.300019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 504/1037] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:41.301200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:41.328438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:41.405255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305221e-73dd-4e22-b985-4362270dcba9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:41.439656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:41.440459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:41.440745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:41.441026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:41.449424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:41.450974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:41.465716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:41.467365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:41.468350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 534/1038] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:51:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:41.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:41.537214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:41.577392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e100-3a3d-4027-961a-8d848f3b34bb None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:41.612972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:41.613642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:41.613926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:41.614205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:41.615055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:41.615367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:41.616809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:41.638287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 505/1039] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:51:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:41.639605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:41.733472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:41.750676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0872-749e-4deb-bdff-956c7c78118a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:41.785956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9a617-597d-498c-a528-f6f4cd525976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:41.786602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9a617-597d-498c-a528-f6f4cd525976 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:41.786913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9a617-597d-498c-a528-f6f4cd525976 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:41.787197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9a617-597d-498c-a528-f6f4cd525976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:41.857985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9a617-597d-498c-a528-f6f4cd525976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:41.859503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9a617-597d-498c-a528-f6f4cd525976 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:41.942681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:41.965224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 535/1040] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:41.966704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.040702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0af4a-0587-42ab-a551-11343ba74795 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.041191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0af4a-0587-42ab-a551-11343ba74795 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.041375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0af4a-0587-42ab-a551-11343ba74795 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.042887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 536/1041] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:42.047796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.048085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.048366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.065633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:42.066809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:42.080380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3a705-2af3-4c3b-a772-13af1f3926f0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.102405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.118301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.119174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.119468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.119748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.131542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.132973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.142909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 537/1042] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:42.155901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:42.156792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:42.157701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:42.157701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:42.157701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:42.157701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:42.157701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 506/1043] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:42.163106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.165939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.270248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7cc1-81f1-4efb-a8c3-7ad48e1285ac None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.270579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fb1b5-9340-44a7-948f-6bc027d5b02b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.285032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.307493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.308191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.308480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.308761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.309019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.309512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:42.309826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.310062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.310451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.310823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.311405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.322189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:42.327325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 538/1044] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:42.328188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:42.330294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:42.331419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:42.332278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:42.337097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 507/1045] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:42.338017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.371388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.437849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864bdad-e8a8-4980-b918-2d5856db6b7a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.448978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76913e2c-6a31-4051-b4ab-8707e6cd6ef9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.474269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.475099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.475385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.475662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.484083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.485669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.489552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.490225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.490508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.490781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.491554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:42.491860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.493238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.504484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 508/1046] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:42.509958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:42.510072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:42.510930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:42.511836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:42.512274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 539/1047] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:42.517501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.554590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.616670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.625175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e0902-bcc3-484f-ae89-63844160478f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.634777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28686cf4-831a-458b-b161-1d5ca543ddd5 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.667586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.668627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.668912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.669187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] PATH_INFO: `/v3/users/8ad20a45d51c4080a35e0cd9956c2048` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.675132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.675987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.676396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.676776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.677674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:42.678093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.679685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.685401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] RBAC: Authorizing `identity:get_user(user_id=8ad20a45d51c4080a35e0cd9956c2048)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.686984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.694600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 509/1048] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/users/8ad20a45d51c4080a35e0cd9956c2048 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:42.695437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.698491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 540/1049] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:42.704097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:42.801350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.809924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c1c6-ad7b-4570-b70f-7999a22a6403 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.826582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2670a51-85bb-4e9d-8bf9-6391e51c255b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:42.846869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.847686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.848036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.848378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.849267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] RBAC: Adding query filter params (role.id=3467f6f457b949a7a92d0e944b1acec3, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=b0208c5889a74870b764c7eb954242bb) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:42.849667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.851229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a51a2e-76cb-4618-b34a-e69b1610e4bb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.867315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 541/1050] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/role_assignments?user.id=b0208c5889a74870b764c7eb954242bb&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=3467f6f457b949a7a92d0e944b1acec3 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:42.879990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:42.880737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.881122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.881650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.911578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:42.968952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:42.970604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:42.991908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1befc9a-4923-4483-972a-431556c0b948 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:42.992237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1befc9a-4923-4483-972a-431556c0b948 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:42.992546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1befc9a-4923-4483-972a-431556c0b948 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:42.994229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 542/1051] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:51:43.000067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.000397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.000716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.024601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:43.025841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:43.104210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 510/1052] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 11:51:43.104417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 543/1053] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:43.105258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:43.123008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:43.168842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:43.216109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:43.230343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9339ecd-a07b-4bda-a1ce-17488554a25f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:43.238560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a88053-1c33-4042-a695-69beeb8f8736 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:43.269585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:43.270710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.271216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.271697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] PATH_INFO: `/v3/services/image` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.272504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:43.274224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:43.276665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:43.277526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.277813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.278096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.279306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c831e30f194753903e3af27b515a0c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:43.280722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:43.282417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 511/1054] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:51:43 2022] GET /identity/v3/projects/d5c831e30f194753903e3af27b515a0c => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:43.288702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:43.290032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 11:51:43.291234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:43.292171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:43.293817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 544/1055] 10.222.0.41 () {56 vars in 1105 bytes} [Mon Jun 6 11:51:43 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:43.299397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:43.397098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700511-7589-475e-89c3-7cc63c579a73 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:43.416231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d2f6-7ecd-4424-b9c7-3002a4e01d30 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:43.417822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:43.434183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:43.435097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.435532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.435959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.436918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=d5c831e30f194753903e3af27b515a0c, user.id=8ad20a45d51c4080a35e0cd9956c2048) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:43.437406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:43.439048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:43.445375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 512/1056] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:43 2022] GET /identity/v3/role_assignments?user.id=8ad20a45d51c4080a35e0cd9956c2048&scope.project.id=d5c831e30f194753903e3af27b515a0c&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:43.453528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:43.454216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.454507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.454783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.456084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] RBAC: Adding query filter params (name=image) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:43.456657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:43.458090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:43.464061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 545/1057] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:43.470042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:43.587197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877f358-9619-4ad7-8eef-3d119b0db45c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:43.623350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:43.624140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.624430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.624709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.625420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] RBAC: Adding query filter params (type=image) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:43.625728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:43.627742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e43f0-bd2b-4e11-9114-233215f27aad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:43.638252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 513/1058] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:43.639099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:43.661299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:43.751155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d6b62-f5c3-4dd8-a0a5-7232eaf1d67d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:43.787369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af697b72-270c-4cb9-bd6b-db7a1ac0ab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:43.788370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af697b72-270c-4cb9-bd6b-db7a1ac0ab9f None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:43.789558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af697b72-270c-4cb9-bd6b-db7a1ac0ab9f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:43.790046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af697b72-270c-4cb9-bd6b-db7a1ac0ab9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:43.791083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af697b72-270c-4cb9-bd6b-db7a1ac0ab9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:43.792669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af697b72-270c-4cb9-bd6b-db7a1ac0ab9f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:43.818917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 546/1059] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:51:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:51:45.138518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960049d8-5208-4a69-a394-8790180778a7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.139071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960049d8-5208-4a69-a394-8790180778a7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.139149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960049d8-5208-4a69-a394-8790180778a7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.140667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 514/1060] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:45.145711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.146096 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.146274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.163497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:45.164697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:45.234119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:45.242988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 547/1061] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:45.264404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960049d8-5208-4a69-a394-8790180778a7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:45.317212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:45.370172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960049d8-5208-4a69-a394-8790180778a7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:45.409493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:45.410306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.410592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.410895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.421431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:45.429563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:45.430473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:45.431454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:45.432849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 515/1062] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 11:51:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:45.436684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:45.541062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6bc19-26e3-4e07-9dcc-18ea8fe58c29 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:45.575797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:45.576386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.576672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.576951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.577700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:45.578013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:45.579441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501efb3f-7c47-44ed-a690-83b370e646af None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:45.593344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 548/1063] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:51:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:45.594295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:45.699772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592e3eb-9c7f-40f9-a5d1-a305b7dafc71 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:45.709578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:45.712560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477f015-2a12-4bea-9ba5-3730852e0640 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.712906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477f015-2a12-4bea-9ba5-3730852e0640 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.713284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477f015-2a12-4bea-9ba5-3730852e0640 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.714982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 549/1064] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:45.720074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.720365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.720645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.740218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:45.741173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:45.741400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:45.742040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.742333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.742615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.760346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:45.761857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:45.781728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:45.782950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:45.784659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:45.784659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:45.784659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:45.784659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:45.784659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 516/1065] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:45.788982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:45.814007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:45.829242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 550/1066] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:45.852415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:45.897748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46ae2-ef26-4970-96c6-e11955e635f4 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:45.933332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:45.933995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:45.934285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:45.934573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:45.935328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:45.935643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:45.937017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:45.950709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 517/1067] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:45.956418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:45.964360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141311a-4fd8-4db3-9ea7-2dfb9c5485c3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:45.978636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.005566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.007148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.007628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.007911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.021784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:46.029422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:46.030359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:46.031259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 551/1068] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:46.037187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.066519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb11f6-0cc2-4c27-8e64-532330a0b8d3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.095471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.106930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.107725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.108017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.108323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.116137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.117601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-542d9e48-e823-4efa-acf1-96a042922202 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:46.131830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:46.132883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 518/1069] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:51:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ca71-958d-4250-8308-aef3d60a554c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.139259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ca71-958d-4250-8308-aef3d60a554c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.140313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ca71-958d-4250-8308-aef3d60a554c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.140313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 519/1070] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:46.140313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ca71-958d-4250-8308-aef3d60a554c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.149172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07497-fdeb-4237-a7f7-5a84e6af1e8e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.184893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.185566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.185860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.186147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.186928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:46.187243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.188625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6973e-0b58-4142-971e-c603afa4f4d8 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.196931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 552/1071] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:46.198890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.199192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.199478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.216729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:46.217909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:46.224456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.245384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ca71-958d-4250-8308-aef3d60a554c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.279255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.279924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.280207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.280489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.281321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:46.281633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.283085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.285058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 553/1072] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:46.285891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.301957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 520/1073] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:46.307366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.338178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.393831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314433c-3ac4-425f-afab-e85a14b82fad None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.406972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791bfd2-f014-472d-8242-d628ebcfdf60 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.427395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.428255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.428541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.428817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] PATH_INFO: `/v3/users/8ad20a45d51c4080a35e0cd9956c2048` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.429996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] RBAC: Authorizing `identity:get_user(user_id=8ad20a45d51c4080a35e0cd9956c2048)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.431483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.433114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 554/1074] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/users/8ad20a45d51c4080a35e0cd9956c2048 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:46.433882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.441814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.442802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.443122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.443399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.450109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.454526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.456674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:46.474179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:46.475141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:46.476023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:46.476023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:46.476023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:46.476023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:46.476023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 521/1075] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:46.476589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.538284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1c61-1f87-4fc2-a303-c9a9cc86ca6a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.548594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.573465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.574048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e53962-53d7-474c-9982-c8075511c3a0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.574155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.574384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.574664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.607775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.608627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.608940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.609220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.610387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c831e30f194753903e3af27b515a0c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.612428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.614114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 522/1076] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/projects/d5c831e30f194753903e3af27b515a0c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:46.614898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.644364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.645783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.672856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.714110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215fc46-5a25-40c7-8816-9f15212a66b8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.744682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 555/1077] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:46.744682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.749828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.749828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.749828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.749828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.750264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:46.750571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.751982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.757471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.763737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 523/1078] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:46.764535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.853497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6adabf-43ef-4e86-afa8-7434e32532a2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.866918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f000a-721b-46fc-b264-d50ac8b870bc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:46.883263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:46.888569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.889452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.889735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.890015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c/users/8ad20a45d51c4080a35e0cd9956c2048/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.891985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c831e30f194753903e3af27b515a0c, user_id=8ad20a45d51c4080a35e0cd9956c2048, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.893688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.901554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:46.902353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:46.902641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:46.902950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:46.906403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 556/1079] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:46 2022] PUT /identity/v3/projects/d5c831e30f194753903e3af27b515a0c/users/8ad20a45d51c4080a35e0cd9956c2048/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:51:46.907239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:46.914468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:46.916045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:46.935798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:46.936783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:46.937729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:46.937729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:46.937729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:46.937729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:46.937729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 524/1080] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:46.943443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:47.015325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:47.053637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c87a-f79f-4914-bf37-570099cd3aac None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:47.064284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e72567-30c7-4528-9acf-773ee7984383 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:47.098056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:47.098896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:47.099187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:47.099467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:47.104264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:47.105119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:47.105568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:47.105980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:47.106875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:47.107472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:47.108909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:47.109174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:47.110447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:47.123486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 525/1081] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:47.125587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:47.126583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:47.128334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:47.131228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 557/1082] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:47.133617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:47.170131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:47.251852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2809341-126d-43ea-a07c-500edddbe651 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:47.291937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:47.292620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:47.292969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:47.293256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:47.294137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:47.294463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:47.296122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71907d5f-a559-40a5-a302-a14cd0a68168 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:47.310676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 526/1083] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:47.317595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:47.424949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc762b-50df-4dc2-a5b2-0475245426a1 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:47.453852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:47.461914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd18c6-1b01-4cf9-81a7-bee05b933c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:47.463064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd18c6-1b01-4cf9-81a7-bee05b933c4a None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:47.463354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd18c6-1b01-4cf9-81a7-bee05b933c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:47.463632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd18c6-1b01-4cf9-81a7-bee05b933c4a None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/b0208c5889a74870b764c7eb954242bb/roles/3467f6f457b949a7a92d0e944b1acec3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:47.465841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd18c6-1b01-4cf9-81a7-bee05b933c4a None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=b0208c5889a74870b764c7eb954242bb, role_id=3467f6f457b949a7a92d0e944b1acec3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:47.467607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd18c6-1b01-4cf9-81a7-bee05b933c4a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:47.481832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 558/1084] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:47 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/b0208c5889a74870b764c7eb954242bb/roles/3467f6f457b949a7a92d0e944b1acec3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:51:49.162182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b3360-e2bb-4c5f-ad45-f939bb3934e5 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.162756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b3360-e2bb-4c5f-ad45-f939bb3934e5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.162756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b3360-e2bb-4c5f-ad45-f939bb3934e5 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.164182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 527/1085] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:49.169603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971d841-9268-4547-8623-4f05edba114c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.170029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971d841-9268-4547-8623-4f05edba114c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.170177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971d841-9268-4547-8623-4f05edba114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.187367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c971d841-9268-4547-8623-4f05edba114c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:49.188567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c971d841-9268-4547-8623-4f05edba114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:49.257188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 559/1086] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:49.286615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b3360-e2bb-4c5f-ad45-f939bb3934e5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:49.378586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:49.390595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b3360-e2bb-4c5f-ad45-f939bb3934e5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:49.426819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:49.427630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.427914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.428188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.438420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:49.445990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:49.447003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:49.447919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 528/1087] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:49.453142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971d841-9268-4547-8623-4f05edba114c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:49.550213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971d841-9268-4547-8623-4f05edba114c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:49.571683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:49.587105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:49.587780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.588070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.588355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.589104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:49.589415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:49.590804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d1364-fb4b-4271-8fbd-92273457d0b9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:49.599274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 560/1088] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:49.604987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:49.704447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fe1f1-53cf-48d4-914d-c74cd4a99e9a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:49.711482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561a40-3582-423d-9a25-ad10aa67fb42 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.711783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561a40-3582-423d-9a25-ad10aa67fb42 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.712085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561a40-3582-423d-9a25-ad10aa67fb42 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.713750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 561/1089] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:49.718778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.719098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.719385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.737103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:49.738475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:49.743156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:49.744034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.744323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.744606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] PATH_INFO: `/v3/users/8ad20a45d51c4080a35e0cd9956c2048` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.745786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] RBAC: Authorizing `identity:get_user(user_id=8ad20a45d51c4080a35e0cd9956c2048)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:49.747274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:49.748918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 529/1090] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:49 2022] GET /identity/v3/users/8ad20a45d51c4080a35e0cd9956c2048 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:49.754911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:49.773024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:49.808703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 562/1091] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:51:49.831698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:49.855480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7da897-f28c-4da1-b408-9c5fd6710e17 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:49.889936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:49.890583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.890891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.891174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.929661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a732095-f007-4288-8436-6129e6f17d2d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:49.942888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:49.959904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:49.961361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:49.965900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:49.966995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:49.967282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:49.967563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:49.978469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:51:49.979505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:51:49.986572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:49.987775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:49.988624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 563/1092] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 11:51:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:49.993667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.068500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 530/1093] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 11:51:50.070643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.094066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a3f62-de3e-4d1c-a083-874a9272b47a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.127330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.127991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.128287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.128569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.129313 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:50.129627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.131032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.138754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 564/1094] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:51:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:50.145053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.167735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeadbc-abd1-4672-a6c4-1a859cc6a8a2 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.179359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:50.186461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:50.203935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.204792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.205461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.205745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.206938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] RBAC: Authorizing `identity:get_project(project_id=d5c831e30f194753903e3af27b515a0c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.208362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.210041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 531/1095] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/projects/d5c831e30f194753903e3af27b515a0c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:50.217113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.246258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e57bf-7996-4696-88e3-bb05d09ed693 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.280263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.281050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.281338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.281619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.292590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.294139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:50.311798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:50.312907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:50.313852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:50.313852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:50.313852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:50.313852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:50.313852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 565/1096] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:50.319512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.320894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1db492-16a9-429c-91d9-b4affa10f99d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.355164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.355809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.356097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.356376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.357195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=d5c831e30f194753903e3af27b515a0c, user.id=8ad20a45d51c4080a35e0cd9956c2048) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:50.357513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.358967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.365352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 532/1097] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/role_assignments?user.id=8ad20a45d51c4080a35e0cd9956c2048&scope.project.id=d5c831e30f194753903e3af27b515a0c&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:50.415696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:50.433820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf878908-64f8-4c49-8e17-972b8637fffe None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.469756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.470438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.471367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.471759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.472500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:50.472811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.474219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.488577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 566/1098] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:51:50.494239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.547222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:50.601442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a15425-8377-44a6-8dad-fb746abb5a76 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.636859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.637654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.637945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.638227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.646125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.647606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:51:50.662581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:50.663801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:51:50.664795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 533/1099] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:51:50.669496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.769413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406631-3ec8-4cb7-8831-f07f8e4220d2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.807280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.807962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.808251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.808533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:50.809377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:50.809692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:50.811180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:50.828745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 567/1100] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:50.834917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:50.936985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e8a3-8a95-45cf-b337-b82e5ba7210b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:50.947790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:50.972887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:50.973537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:50.973819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:50.974096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:51.042595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:51.044099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:51.140545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 534/1101] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:51.146749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:51.150568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:51.247082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95180-b571-4e6a-aad7-72ff27a78531 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:51.279551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:51.280337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:51.280620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:51.280897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:51.291118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:51.292571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:51:51.309389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:51.310434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:51.311419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:51:51.311419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:51:51.311419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:51:51.311419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:51.311419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 568/1102] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:51.316988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:51.328194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:51.414525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cae44-7b72-4d4c-bb65-8bcc5bbaa763 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:51.448210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:51.448871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:51.449157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:51.449436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:51.450146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:51.450454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:51.451876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:51.463628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 535/1103] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:51.469345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:51.527118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:51.571044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144a58-073f-4625-b574-9e6b5cac3497 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:51.604752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:51.605545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:51.605832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:51.606109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:51.613244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:51.614697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:51:51.628704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:51:51.629866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:51:51.630772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 569/1104] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:51:51.635704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:51.703788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:51.735274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f7b7d-3fdd-4dbb-b2d9-817914a9c345 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:51.768831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:51.769484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:51.769772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:51.770055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:51.770832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:51.771169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:51.772592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c42e03-eaec-4f72-ba5e-b7153a245058 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:51.784571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 536/1105] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:51.790258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:51.889953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657c025-009e-4a3d-9126-5462db36d8e3 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:51.905522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:51.923562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:51.924289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:51.924628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:51.924909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:51.925739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] RBAC: Adding query filter params (role.id=3467f6f457b949a7a92d0e944b1acec3, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=b0208c5889a74870b764c7eb954242bb) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:51.926053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:51.927545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec5dda-6430-45bc-9dae-628af962334f None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:51.933342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 570/1106] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:51 2022] GET /identity/v3/role_assignments?user.id=b0208c5889a74870b764c7eb954242bb&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=3467f6f457b949a7a92d0e944b1acec3 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:52.625626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b1cd4-8cad-4c2c-88f2-871e33bc188b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:52.626141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b1cd4-8cad-4c2c-88f2-871e33bc188b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:52.626230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b1cd4-8cad-4c2c-88f2-871e33bc188b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:52.628199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 537/1107] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:52.633491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:52.633923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:52.634126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:52.651792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:52.652979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:52.723435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 571/1108] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:52.748853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b1cd4-8cad-4c2c-88f2-871e33bc188b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:52.851258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b1cd4-8cad-4c2c-88f2-871e33bc188b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:52.880182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:52.886193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae26a0e-f115-4b3a-8202-126d6ecee070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:52.887006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae26a0e-f115-4b3a-8202-126d6ecee070 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:52.887293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae26a0e-f115-4b3a-8202-126d6ecee070 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:52.887569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae26a0e-f115-4b3a-8202-126d6ecee070 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:52.888729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae26a0e-f115-4b3a-8202-126d6ecee070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:52.890161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae26a0e-f115-4b3a-8202-126d6ecee070 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:52.891803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 538/1109] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:51:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:52.898100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:52.994720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ebe38-434f-43f3-9bf7-8633143b7e7b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:53.006136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:53.030028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af52830-b962-4bfd-bd65-b7809738a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:53.030692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af52830-b962-4bfd-bd65-b7809738a78c None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:53.031008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af52830-b962-4bfd-bd65-b7809738a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:53.031288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af52830-b962-4bfd-bd65-b7809738a78c None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:53.032107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af52830-b962-4bfd-bd65-b7809738a78c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:53.033550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af52830-b962-4bfd-bd65-b7809738a78c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:53.053538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 572/1110] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:51:52 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:51:55.340189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f0da1-607b-40e1-b92a-5ed2f27bbff2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:55.340727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f0da1-607b-40e1-b92a-5ed2f27bbff2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:55.340727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f0da1-607b-40e1-b92a-5ed2f27bbff2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:55.342391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 539/1111] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:55.348434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:55.348829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:55.349001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:55.365688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:55.366897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:55.431255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 573/1112] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:55.454381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f0da1-607b-40e1-b92a-5ed2f27bbff2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:55.553014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f0da1-607b-40e1-b92a-5ed2f27bbff2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:55.584921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:55.585761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:55.586050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:55.586329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] PATH_INFO: `/v3/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:55.587794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] RBAC: Authorizing `identity:get_role(role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:55.589163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:55.590553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 540/1113] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:55 2022] GET /identity/v3/roles/36fb427222cd413e808b7d4028e53faf => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:55.597219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:55.687678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:55.694271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6553d3-3a3c-4239-80fe-b99dd8c7ed47 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:55.728033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:55.728905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:55.729295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:55.729551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] PATH_INFO: `/v3/users/8426ade8e5f04ae3bf91c359191d29f4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:55.743315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] RBAC: Authorizing `identity:get_user(user_id=8426ade8e5f04ae3bf91c359191d29f4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:55.744801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:55.746460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 574/1114] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:55 2022] GET /identity/v3/users/8426ade8e5f04ae3bf91c359191d29f4 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:55.752486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:55.815764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:55.849525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe092ab-57a8-4b6f-8b41-c5d42c70962e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:55.882180 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:55.882815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:55.883133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:55.883411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:55.947517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:55.948952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:56.010697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:56.045528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 541/1115] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:51:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:51:56.052028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:56.148463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954944a-4bc3-45be-a384-ed649f6949ad None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:56.181301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9c0c0-dfb9-4bf9-ae48-796b087ae769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:56.182147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c0c0-dfb9-4bf9-ae48-796b087ae769 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:56.182434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c0c0-dfb9-4bf9-ae48-796b087ae769 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:56.182717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c0c0-dfb9-4bf9-ae48-796b087ae769 None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:56.183918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c0c0-dfb9-4bf9-ae48-796b087ae769 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c831e30f194753903e3af27b515a0c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:56.185367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c0c0-dfb9-4bf9-ae48-796b087ae769 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:56.187087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 575/1116] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:51:56 2022] GET /identity/v3/projects/d5c831e30f194753903e3af27b515a0c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:56.193106 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:56.290462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c447526-722e-46e9-b865-f5e839957914 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:56.322958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:56.323603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:56.323886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:56.324163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:56.324975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] RBAC: Adding query filter params (role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=d5c831e30f194753903e3af27b515a0c, user.id=8426ade8e5f04ae3bf91c359191d29f4) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:51:56.325285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:56.326705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32936b-1e41-4b55-a1a4-dc2adf42defe None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:56.332644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 542/1117] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:51:56 2022] GET /identity/v3/role_assignments?user.id=8426ade8e5f04ae3bf91c359191d29f4&scope.project.id=d5c831e30f194753903e3af27b515a0c&role.id=36fb427222cd413e808b7d4028e53faf => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:58.518128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00057117-856e-4f4e-a999-5c03ed213f3f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:58.518733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00057117-856e-4f4e-a999-5c03ed213f3f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:58.518733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00057117-856e-4f4e-a999-5c03ed213f3f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:58.520096 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 576/1118] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:51:58.525341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:58.525745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:58.525902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:58.542927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:51:58.544107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:51:58.568325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:58.612232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 543/1119] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:51:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:51:58.634827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00057117-856e-4f4e-a999-5c03ed213f3f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:58.659592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:58.735127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00057117-856e-4f4e-a999-5c03ed213f3f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:58.768777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:58.769566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:58.769852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:58.770132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] PATH_INFO: `/v3/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:58.771642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] RBAC: Authorizing `identity:get_role(role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:58.773044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:58.774407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 577/1120] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:58 2022] GET /identity/v3/roles/36fb427222cd413e808b7d4028e53faf => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:58.780084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:58.879146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01d003-2845-4610-b4e8-d4a37cd754ea None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:58.914465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:58.915370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:58.915655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:58.915933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] PATH_INFO: `/v3/users/8426ade8e5f04ae3bf91c359191d29f4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:58.917114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] RBAC: Authorizing `identity:get_user(user_id=8426ade8e5f04ae3bf91c359191d29f4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:58.918545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:58.920195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 544/1121] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:51:58 2022] GET /identity/v3/users/8426ade8e5f04ae3bf91c359191d29f4 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:58.926767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:59.014695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:59.025148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db405c-62de-4d5b-a4e3-c7229f5e5284 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:59.058329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c9cbb-ff2e-478f-bc78-d80541f39c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:59.059197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c9cbb-ff2e-478f-bc78-d80541f39c80 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:59.059484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c9cbb-ff2e-478f-bc78-d80541f39c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:59.059764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c9cbb-ff2e-478f-bc78-d80541f39c80 None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:59.060923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c9cbb-ff2e-478f-bc78-d80541f39c80 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c831e30f194753903e3af27b515a0c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:59.062361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c9cbb-ff2e-478f-bc78-d80541f39c80 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:59.064067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 578/1122] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:51:58 2022] GET /identity/v3/projects/d5c831e30f194753903e3af27b515a0c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:51:59.069967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:51:59.097937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:51:59.166544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26ecd-3be7-41f9-bb08-4c0032325fda None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:51:59.201874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a190dc-2cbf-44ab-a330-abcb5667fb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:51:59.202974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a190dc-2cbf-44ab-a330-abcb5667fb7d None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:51:59.203267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a190dc-2cbf-44ab-a330-abcb5667fb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:51:59.203541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a190dc-2cbf-44ab-a330-abcb5667fb7d None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c/users/8426ade8e5f04ae3bf91c359191d29f4/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:51:59.206139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a190dc-2cbf-44ab-a330-abcb5667fb7d None admin] RBAC: Authorizing `identity:create_grant(project_id=d5c831e30f194753903e3af27b515a0c, user_id=8426ade8e5f04ae3bf91c359191d29f4, role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:51:59.207602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a190dc-2cbf-44ab-a330-abcb5667fb7d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:51:59.220382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 545/1123] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:51:59 2022] PUT /identity/v3/projects/d5c831e30f194753903e3af27b515a0c/users/8426ade8e5f04ae3bf91c359191d29f4/roles/36fb427222cd413e808b7d4028e53faf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:52:01.428202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638cc71-12ea-493c-9dc1-c917e147f984 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:01.428728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638cc71-12ea-493c-9dc1-c917e147f984 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:01.428842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638cc71-12ea-493c-9dc1-c917e147f984 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:01.430127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 579/1124] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:01.435524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:01.435917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:01.436090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:01.452377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:01.453553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:01.518189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 546/1125] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:01.541979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638cc71-12ea-493c-9dc1-c917e147f984 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:01.641287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638cc71-12ea-493c-9dc1-c917e147f984 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:01.675453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:01.676289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:01.676574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:01.676852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] PATH_INFO: `/v3/roles/36fb427222cd413e808b7d4028e53faf` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:01.678373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] RBAC: Authorizing `identity:get_role(role_id=36fb427222cd413e808b7d4028e53faf)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:01.679732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:01.681091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 580/1126] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:01 2022] GET /identity/v3/roles/36fb427222cd413e808b7d4028e53faf => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:01.686892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:01.704747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:01.783959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d296609-affe-40a7-83d4-d18c169bf986 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:01.816457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:01.817318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:01.817602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:01.817877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] PATH_INFO: `/v3/users/8426ade8e5f04ae3bf91c359191d29f4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:01.819059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] RBAC: Authorizing `identity:get_user(user_id=8426ade8e5f04ae3bf91c359191d29f4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:01.820476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:01.822085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 547/1127] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:01 2022] GET /identity/v3/users/8426ade8e5f04ae3bf91c359191d29f4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:01.827751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:01.855080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:01.926149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee285ce8-fedd-42be-8321-5b58b3d3ba83 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:01.959196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:01.959771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:01.960060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:01.960342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:02.025615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:02.027110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:02.052564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:02.122564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 581/1128] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:52:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:52:02.128402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:02.209775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:02.224722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02aacc3-89b4-4204-b990-bd4ea5b4138a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:02.258067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efad09d-2781-4b9d-9b39-a973ac0615b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:02.258942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad09d-2781-4b9d-9b39-a973ac0615b2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:02.259232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad09d-2781-4b9d-9b39-a973ac0615b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:02.259512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad09d-2781-4b9d-9b39-a973ac0615b2 None admin] PATH_INFO: `/v3/projects/d5c831e30f194753903e3af27b515a0c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:02.260664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad09d-2781-4b9d-9b39-a973ac0615b2 None admin] RBAC: Authorizing `identity:get_project(project_id=d5c831e30f194753903e3af27b515a0c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:02.262093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad09d-2781-4b9d-9b39-a973ac0615b2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:02.263796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 548/1129] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:02 2022] GET /identity/v3/projects/d5c831e30f194753903e3af27b515a0c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:02.270062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:02.367860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895b00a-876c-44b0-9fb2-86beba832e49 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:02.392224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:02.401022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:02.401669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:02.401957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:02.402236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:02.403083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] RBAC: Adding query filter params (role.id=36fb427222cd413e808b7d4028e53faf, scope.project.id=d5c831e30f194753903e3af27b515a0c, user.id=8426ade8e5f04ae3bf91c359191d29f4) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:02.403398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:02.404836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067170e-e4dc-41ed-ace7-f7681bbbea7c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:02.411090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 582/1130] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:52:02 2022] GET /identity/v3/role_assignments?user.id=8426ade8e5f04ae3bf91c359191d29f4&scope.project.id=d5c831e30f194753903e3af27b515a0c&role.id=36fb427222cd413e808b7d4028e53faf => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:04.656024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec282-0aae-4e1c-80d8-e536995ac147 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:04.656584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec282-0aae-4e1c-80d8-e536995ac147 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:04.656584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec282-0aae-4e1c-80d8-e536995ac147 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:04.658467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 549/1131] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:04.664047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:04.664448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:04.664612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:04.681840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:04.683053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:04.748920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 583/1132] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:04.771961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ec282-0aae-4e1c-80d8-e536995ac147 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:04.874415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ec282-0aae-4e1c-80d8-e536995ac147 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:04.910051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533dd3-adf0-40fe-8d83-c3b9ac55cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:04.910771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533dd3-adf0-40fe-8d83-c3b9ac55cc29 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:04.911092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533dd3-adf0-40fe-8d83-c3b9ac55cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:04.911372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533dd3-adf0-40fe-8d83-c3b9ac55cc29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:04.912535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533dd3-adf0-40fe-8d83-c3b9ac55cc29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:04.913969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533dd3-adf0-40fe-8d83-c3b9ac55cc29 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:04.915621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 550/1133] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:52:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:04.921557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:05.018640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c07b4-37e0-419e-942b-90fcfa9c3431 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:05.038621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:05.052808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288446ae-85bd-4f91-83db-ad80de36a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:05.053471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288446ae-85bd-4f91-83db-ad80de36a5d6 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:05.053751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288446ae-85bd-4f91-83db-ad80de36a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:05.054032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288446ae-85bd-4f91-83db-ad80de36a5d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:05.054732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288446ae-85bd-4f91-83db-ad80de36a5d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:05.056205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288446ae-85bd-4f91-83db-ad80de36a5d6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:05.080344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 584/1134] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:52:04 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:52:07.322830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fca30d-60a8-42ac-a8e5-fff9f10f0a00 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:07.323360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fca30d-60a8-42ac-a8e5-fff9f10f0a00 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:07.323448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fca30d-60a8-42ac-a8e5-fff9f10f0a00 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:07.324771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 551/1135] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:07.330599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:07.331211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:07.331211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:07.348457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:07.349646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:07.413340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 585/1136] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:52:07.438045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fca30d-60a8-42ac-a8e5-fff9f10f0a00 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:07.463000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:07.539868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fca30d-60a8-42ac-a8e5-fff9f10f0a00 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:07.572558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38c828-3522-4a43-80b4-df2fa0657391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:07.573389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c828-3522-4a43-80b4-df2fa0657391 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:07.573628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c828-3522-4a43-80b4-df2fa0657391 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:07.573906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c828-3522-4a43-80b4-df2fa0657391 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:07.575097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38c828-3522-4a43-80b4-df2fa0657391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:07.576635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38c828-3522-4a43-80b4-df2fa0657391 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:07.578232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 552/1137] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:52:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:07.584270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:07.681181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f045f73-b381-4df4-94d1-a5a8353f91bc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:07.682672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:07.715788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3d6c1-1126-438a-86f2-33606c6f7b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:07.716446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3d6c1-1126-438a-86f2-33606c6f7b95 None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:07.716983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3d6c1-1126-438a-86f2-33606c6f7b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:07.717263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3d6c1-1126-438a-86f2-33606c6f7b95 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:07.717955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3d6c1-1126-438a-86f2-33606c6f7b95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:07.719422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3d6c1-1126-438a-86f2-33606c6f7b95 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:07.740932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 586/1138] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:52:07 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:52:09.977002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bac4b-ceff-425d-a73b-b251b441aa4c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:09.977557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bac4b-ceff-425d-a73b-b251b441aa4c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:09.977557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bac4b-ceff-425d-a73b-b251b441aa4c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:09.978956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 553/1139] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:09.984041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:09.984431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:09.984605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:10.001659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:10.002946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:10.067795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 587/1140] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:10.091488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bac4b-ceff-425d-a73b-b251b441aa4c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:10.179508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:10.190690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bac4b-ceff-425d-a73b-b251b441aa4c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:10.224352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:10.225165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:10.225417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:10.225695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:10.235463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:52:10.243524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:10.244630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:52:10.245677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 554/1141] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:10.250652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:10.336518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:10.349787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0ffe4-4bc1-4705-b98e-cb937e221956 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:10.382476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:10.383159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:10.383448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:10.383730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:10.384475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:10.384787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:10.386165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:10.394577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 588/1142] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:10.400591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:10.498176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95439d7d-2d67-4099-9bed-aff6dea6b4fc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:10.530557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:10.531448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:10.531736 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:10.532013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] PATH_INFO: `/v3/users/7d27e3ca4ac64680a5c237166f550da9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:10.544757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] RBAC: Authorizing `identity:get_user(user_id=7d27e3ca4ac64680a5c237166f550da9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:10.546196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:10.547856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 555/1143] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:10 2022] GET /identity/v3/users/7d27e3ca4ac64680a5c237166f550da9 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:10.554280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:10.651621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861bcc5-dfa0-45e8-a32f-9cb68b9973ba None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:10.684244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:10.684878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:10.685161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:10.685437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:10.703031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:10.752686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:10.754080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:10.848685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 589/1144] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:52:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:52:10.854552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:10.865895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:10.952203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f57fc-d36d-467f-baed-dfa18741d68b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:10.985756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeefc8-e9a7-4d8d-a6c4-ae262244fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:10.986639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeefc8-e9a7-4d8d-a6c4-ae262244fb42 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:10.986946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeefc8-e9a7-4d8d-a6c4-ae262244fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:10.987226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeefc8-e9a7-4d8d-a6c4-ae262244fb42 None admin] PATH_INFO: `/v3/projects/4b12836c0f8d45328476e024593f4c9d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:10.988384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeefc8-e9a7-4d8d-a6c4-ae262244fb42 None admin] RBAC: Authorizing `identity:get_project(project_id=4b12836c0f8d45328476e024593f4c9d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:10.989812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeefc8-e9a7-4d8d-a6c4-ae262244fb42 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:10.991502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 556/1145] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:10 2022] GET /identity/v3/projects/4b12836c0f8d45328476e024593f4c9d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:10.997625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:11.046638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:11.095141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cac22-0e7c-403b-9fe9-9dd9001cd80a None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:11.128089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:11.128731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:11.129019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:11.129301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:11.130124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=4b12836c0f8d45328476e024593f4c9d, user.id=7d27e3ca4ac64680a5c237166f550da9) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:11.130440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:11.131983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59561a10-0646-49c9-9325-25605c05afa9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:11.137300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 590/1146] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:52:10 2022] GET /identity/v3/role_assignments?user.id=7d27e3ca4ac64680a5c237166f550da9&scope.project.id=4b12836c0f8d45328476e024593f4c9d&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:13.416998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f02a-ae07-4ae7-b824-1de84c58197b None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:13.417546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f02a-ae07-4ae7-b824-1de84c58197b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:13.417546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f02a-ae07-4ae7-b824-1de84c58197b None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:13.418938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 557/1147] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:13.424048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:13.424424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:13.424612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:13.440958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:13.442142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:13.507024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 591/1148] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:13.529987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f02a-ae07-4ae7-b824-1de84c58197b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:13.605981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:13.629953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f02a-ae07-4ae7-b824-1de84c58197b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:13.675758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:13.676470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:13.676759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:13.677043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:13.690399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:52:13.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:13.700145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:52:13.701007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 558/1149] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:52:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:13.706315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:13.752354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:13.805607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52377cd9-5bec-4325-8105-d50f8e3115f8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:13.840755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:13.841422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:13.841720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:13.842257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:13.843028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:13.843346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:13.844731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:13.853276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 592/1150] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:52:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:13.859138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:13.959094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47100052-89a3-4f0e-9c09-05e4a376ce20 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:13.993925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:13.994791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:13.995098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:13.995370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] PATH_INFO: `/v3/users/7d27e3ca4ac64680a5c237166f550da9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:13.996542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] RBAC: Authorizing `identity:get_user(user_id=7d27e3ca4ac64680a5c237166f550da9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:13.997988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:13.999644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 559/1151] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:13 2022] GET /identity/v3/users/7d27e3ca4ac64680a5c237166f550da9 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:14.005930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:14.104553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb910f6-8093-45e0-8d34-3c41e6f4526b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:14.111192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:14.137031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8396-f1a2-4306-bd2e-c309f0c72f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:14.137870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8396-f1a2-4306-bd2e-c309f0c72f0c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:14.138155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8396-f1a2-4306-bd2e-c309f0c72f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:14.138433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8396-f1a2-4306-bd2e-c309f0c72f0c None admin] PATH_INFO: `/v3/projects/4b12836c0f8d45328476e024593f4c9d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:14.139762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8396-f1a2-4306-bd2e-c309f0c72f0c None admin] RBAC: Authorizing `identity:get_project(project_id=4b12836c0f8d45328476e024593f4c9d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:14.141201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8396-f1a2-4306-bd2e-c309f0c72f0c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:14.143561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 593/1152] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:14 2022] GET /identity/v3/projects/4b12836c0f8d45328476e024593f4c9d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:14.149497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:14.153245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:14.246784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092124e1-4da2-4930-99f8-8c83740ea59c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:14.279894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b9a70-84f9-47a2-8d3c-ecd6c9d6cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:14.280776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9a70-84f9-47a2-8d3c-ecd6c9d6cf04 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:14.281063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9a70-84f9-47a2-8d3c-ecd6c9d6cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:14.281345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9a70-84f9-47a2-8d3c-ecd6c9d6cf04 None admin] PATH_INFO: `/v3/projects/4b12836c0f8d45328476e024593f4c9d/users/7d27e3ca4ac64680a5c237166f550da9/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:14.284020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9a70-84f9-47a2-8d3c-ecd6c9d6cf04 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b12836c0f8d45328476e024593f4c9d, user_id=7d27e3ca4ac64680a5c237166f550da9, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:14.285456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9a70-84f9-47a2-8d3c-ecd6c9d6cf04 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:14.299498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 560/1153] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:52:14 2022] PUT /identity/v3/projects/4b12836c0f8d45328476e024593f4c9d/users/7d27e3ca4ac64680a5c237166f550da9/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:52:16.615441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ed05d-f3be-4151-a405-7a66f2693f5b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:16.615994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ed05d-f3be-4151-a405-7a66f2693f5b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:16.615994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ed05d-f3be-4151-a405-7a66f2693f5b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:16.617359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 594/1154] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:16.622582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:16.623029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:16.623173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:16.640224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:16.641422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:16.698445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:16.708062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 561/1155] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:52:16.732360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ed05d-f3be-4151-a405-7a66f2693f5b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:16.833688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ed05d-f3be-4151-a405-7a66f2693f5b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:16.866650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:16.867394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:16.867680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:16.867965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:16.878966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:52:16.886270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:16.887369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:52:16.888412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 595/1156] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:52:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:16.894098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:16.998229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d0f-8f6c-4c11-a9ea-4d743a6b631a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:17.031909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:17.032577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:17.032865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:17.033348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:17.034113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:17.034427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:17.035830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:17.043692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 562/1157] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:52:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:52:17.049975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:17.083961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:17.148885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87435b9a-b9d2-4b51-bd44-8a0abaaee8ad None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:17.181356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:17.182207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:17.182490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:17.182767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] PATH_INFO: `/v3/users/7d27e3ca4ac64680a5c237166f550da9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:17.183953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] RBAC: Authorizing `identity:get_user(user_id=7d27e3ca4ac64680a5c237166f550da9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:17.185381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:17.187013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 596/1158] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:17 2022] GET /identity/v3/users/7d27e3ca4ac64680a5c237166f550da9 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:17.193475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:17.248798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:17.292756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c20e36-00aa-4906-bdc8-0e289fcf84a9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:17.326033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:17.326669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:17.326989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:17.327275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:17.393865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:17.395347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:17.447130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:17.490386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 563/1159] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:52:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:52:17.496175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:17.595338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccd32b-3e6b-47db-8977-e915d59e10f2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:17.597345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:17.630703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7605cf-2c6e-4a03-a608-0ed130b81080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:17.631570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7605cf-2c6e-4a03-a608-0ed130b81080 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:17.631849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7605cf-2c6e-4a03-a608-0ed130b81080 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:17.632121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7605cf-2c6e-4a03-a608-0ed130b81080 None admin] PATH_INFO: `/v3/projects/4b12836c0f8d45328476e024593f4c9d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:17.633272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7605cf-2c6e-4a03-a608-0ed130b81080 None admin] RBAC: Authorizing `identity:get_project(project_id=4b12836c0f8d45328476e024593f4c9d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:17.634704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7605cf-2c6e-4a03-a608-0ed130b81080 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:17.636422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 597/1160] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:17 2022] GET /identity/v3/projects/4b12836c0f8d45328476e024593f4c9d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:17.642520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:17.740383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b1650-0a57-4ebd-8be1-2da7ed12909b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:17.773269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:17.773911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:17.774196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:17.774504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:17.775353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=4b12836c0f8d45328476e024593f4c9d, user.id=7d27e3ca4ac64680a5c237166f550da9) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:17.775699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:17.777144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db562ec-d290-4f98-ad52-71e6e2059f58 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:17.782880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 564/1161] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:52:17 2022] GET /identity/v3/role_assignments?user.id=7d27e3ca4ac64680a5c237166f550da9&scope.project.id=4b12836c0f8d45328476e024593f4c9d&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:20.028953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c681a-15d2-4f62-a3bc-fb0eb48700ad None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:20.028953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c681a-15d2-4f62-a3bc-fb0eb48700ad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:20.028953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c681a-15d2-4f62-a3bc-fb0eb48700ad None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:20.029702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 598/1162] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:20.035513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:20.035944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:20.036082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:20.052353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:20.053549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:20.067201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:20.118413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 565/1163] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:52:20.140185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c681a-15d2-4f62-a3bc-fb0eb48700ad None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:20.241347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c681a-15d2-4f62-a3bc-fb0eb48700ad None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:20.274015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:20.274590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:20.274900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:20.275180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:20.312385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:20.340505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:20.341953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:20.434959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 599/1164] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:52:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:52:20.441381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:20.536213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2d653-1c3e-4bc5-9c7d-528ce22c89b8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:20.568474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:20.569253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:20.569534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:20.569808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:20.576233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:20.577655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d657e306-984c-46f2-9bb4-ac0d8b2497f1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:20.591369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:20.592882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:52:20.594008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 566/1165] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:52:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:20.599093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:20.667670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:20.698345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58286-4d02-4055-99a1-6c7e943ea043 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:20.732197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:20.732850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:20.733140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:20.733424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:20.734139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:20.734453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:20.735867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bc8b0-0af3-43ce-8ba6-8c1f9c2bf345 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:20.742924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 600/1166] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 11:52:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:22.970873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ec19e-3a84-4612-9141-22650bc54e5a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:22.971429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ec19e-3a84-4612-9141-22650bc54e5a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:22.971536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ec19e-3a84-4612-9141-22650bc54e5a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:22.972829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 567/1167] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:22.977945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661b28d-6428-470d-b787-fece6a50d737 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:22.978329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661b28d-6428-470d-b787-fece6a50d737 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:22.978510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661b28d-6428-470d-b787-fece6a50d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:22.995328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7661b28d-6428-470d-b787-fece6a50d737 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:22.996494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7661b28d-6428-470d-b787-fece6a50d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:23.062312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 601/1168] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:23.085394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ec19e-3a84-4612-9141-22650bc54e5a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:23.098371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:23.184795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ec19e-3a84-4612-9141-22650bc54e5a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:23.216588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089847d0-7352-4d20-acf6-cca66d5b5ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:23.217231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089847d0-7352-4d20-acf6-cca66d5b5ae7 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:23.217778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089847d0-7352-4d20-acf6-cca66d5b5ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:23.218050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089847d0-7352-4d20-acf6-cca66d5b5ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:23.218765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089847d0-7352-4d20-acf6-cca66d5b5ae7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:23.220142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089847d0-7352-4d20-acf6-cca66d5b5ae7 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:23.236858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 568/1169] 10.222.0.41 () {60 vars in 1138 bytes} [Mon Jun 6 11:52:23 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:52:25.501084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81b3a9-0fc8-4874-8cc7-deb177641c11 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:25.501675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81b3a9-0fc8-4874-8cc7-deb177641c11 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:25.501811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81b3a9-0fc8-4874-8cc7-deb177641c11 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:25.503207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 602/1170] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:25.508497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:25.508914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:25.509053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:25.525844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:25.527053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:25.587986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:25.593269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 569/1171] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:25.617064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81b3a9-0fc8-4874-8cc7-deb177641c11 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:25.678900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:25.717161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81b3a9-0fc8-4874-8cc7-deb177641c11 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:25.754909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a20b0-9939-4d06-bcfa-1bb51c2c24c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:25.755705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a20b0-9939-4d06-bcfa-1bb51c2c24c7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:25.755988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a20b0-9939-4d06-bcfa-1bb51c2c24c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:25.756270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a20b0-9939-4d06-bcfa-1bb51c2c24c7 None admin] PATH_INFO: `/v3/domains/3b09f24b6f504dc6960d8b32b0769f77` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:25.766365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a20b0-9939-4d06-bcfa-1bb51c2c24c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3b09f24b6f504dc6960d8b32b0769f77)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:25.769044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a20b0-9939-4d06-bcfa-1bb51c2c24c7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:25.770675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 603/1172] 10.222.0.41 () {56 vars in 1183 bytes} [Mon Jun 6 11:52:25 2022] GET /identity/v3/domains/3b09f24b6f504dc6960d8b32b0769f77 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:25.776442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:25.873854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf752-e90f-4f6f-b0f6-221d6b4434b9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:25.907943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1427505-0e96-4b8d-ad26-231b9b161b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:25.908976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1427505-0e96-4b8d-ad26-231b9b161b44 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:25.909261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1427505-0e96-4b8d-ad26-231b9b161b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:25.909538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1427505-0e96-4b8d-ad26-231b9b161b44 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:25.910227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1427505-0e96-4b8d-ad26-231b9b161b44 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:25.911668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1427505-0e96-4b8d-ad26-231b9b161b44 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:25.935734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 570/1173] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:52:25 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 11:52:28.212329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d365a-7321-44b6-b6fe-d477c041bd7d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:28.212847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d365a-7321-44b6-b6fe-d477c041bd7d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:28.212953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d365a-7321-44b6-b6fe-d477c041bd7d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:28.214272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 604/1174] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:28.219781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:28.220198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:28.220346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:28.238015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:28.239215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:28.284838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:28.305629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 571/1175] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:28.330729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d365a-7321-44b6-b6fe-d477c041bd7d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:28.429755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d365a-7321-44b6-b6fe-d477c041bd7d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:28.452436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:28.463309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0e6b2-fd00-498a-8f1a-5cce258a8971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:28.464166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0e6b2-fd00-498a-8f1a-5cce258a8971 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:28.464456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0e6b2-fd00-498a-8f1a-5cce258a8971 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:28.464740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0e6b2-fd00-498a-8f1a-5cce258a8971 None admin] PATH_INFO: `/v3/domains/3b09f24b6f504dc6960d8b32b0769f77` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:28.465916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0e6b2-fd00-498a-8f1a-5cce258a8971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3b09f24b6f504dc6960d8b32b0769f77)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:28.467408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0e6b2-fd00-498a-8f1a-5cce258a8971 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:28.469288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 605/1176] 10.222.0.41 () {56 vars in 1183 bytes} [Mon Jun 6 11:52:28 2022] GET /identity/v3/domains/3b09f24b6f504dc6960d8b32b0769f77 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:28.474971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:28.569924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aba57-087c-41e5-823e-0e5faee843ec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:28.602639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b109f-17f3-4203-9a22-820537d46c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:28.603328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b109f-17f3-4203-9a22-820537d46c87 None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:28.603618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b109f-17f3-4203-9a22-820537d46c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:28.603898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b109f-17f3-4203-9a22-820537d46c87 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:28.604595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b109f-17f3-4203-9a22-820537d46c87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:28.606013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b109f-17f3-4203-9a22-820537d46c87 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:28.625348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 572/1177] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:52:28 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:52:30.922055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c3672-a381-4630-9a8a-f840eb1fd6a0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:30.922618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c3672-a381-4630-9a8a-f840eb1fd6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:30.922618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c3672-a381-4630-9a8a-f840eb1fd6a0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:30.924658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 606/1178] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:30.929782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:30.930221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:30.930328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:30.946833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:30.948037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:30.976988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:31.017998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 573/1179] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:31.041975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c3672-a381-4630-9a8a-f840eb1fd6a0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:31.141834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c3672-a381-4630-9a8a-f840eb1fd6a0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:31.175441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:31.176152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:31.176436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:31.176716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:31.186801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:52:31.194266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:31.195422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:52:31.196665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 607/1180] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:31.201352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:31.298474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56c60c-1f4f-41d5-b303-0b7018061809 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:31.331163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:31.331808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:31.332091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:31.332367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:31.333099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:31.333404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:31.334763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:31.342563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 574/1181] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:52:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:31.349226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:31.400369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:31.447145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382f0ef-dd26-49cf-aed1-1f2483acc598 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:31.481518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:31.482362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:31.482646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:31.482949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] PATH_INFO: `/v3/users/9d4e11745b354abb81c487e0d7789a8c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:31.495712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] RBAC: Authorizing `identity:get_user(user_id=9d4e11745b354abb81c487e0d7789a8c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:31.497167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:31.498790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 608/1182] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:31 2022] GET /identity/v3/users/9d4e11745b354abb81c487e0d7789a8c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:31.504687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:31.535765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:31.611824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61c345-2ebe-42a0-b0ee-a4cace1c233c None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:31.645052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:31.645686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:31.645970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:31.646248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:31.712305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:31.713680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:31.741727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:31.810169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 575/1183] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:52:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:52:31.816799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:31.914053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6614fa-12e2-4ddc-b661-9f61a11d6b52 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:31.924454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:31.948019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abce351-834f-43e7-b700-ac0cd27fc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:31.948864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abce351-834f-43e7-b700-ac0cd27fc792 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:31.949148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abce351-834f-43e7-b700-ac0cd27fc792 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:31.949427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abce351-834f-43e7-b700-ac0cd27fc792 None admin] PATH_INFO: `/v3/projects/c30a3afb7c804f759834d45e9bef334d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:31.950588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abce351-834f-43e7-b700-ac0cd27fc792 None admin] RBAC: Authorizing `identity:get_project(project_id=c30a3afb7c804f759834d45e9bef334d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:31.952071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abce351-834f-43e7-b700-ac0cd27fc792 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:31.953754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 609/1184] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:31 2022] GET /identity/v3/projects/c30a3afb7c804f759834d45e9bef334d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:31.960363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:32.056340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fb79a-c1c7-4913-86d1-7fd2b1c6862e None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:32.083821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:32.089966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:32.090614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:32.090937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:32.091215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:32.092034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=c30a3afb7c804f759834d45e9bef334d, user.id=9d4e11745b354abb81c487e0d7789a8c) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:32.092344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:32.094033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8329575-3259-40e9-ae4c-3d4f827ee5ff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:32.099404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 576/1185] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:52:31 2022] GET /identity/v3/role_assignments?user.id=9d4e11745b354abb81c487e0d7789a8c&scope.project.id=c30a3afb7c804f759834d45e9bef334d&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:34.305987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e01a8-da39-44a7-b01f-845fb1065d72 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:34.306523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e01a8-da39-44a7-b01f-845fb1065d72 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:34.306523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e01a8-da39-44a7-b01f-845fb1065d72 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:34.307945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 610/1186] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:34.313469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:34.313890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:34.314018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:34.331863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:34.333056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:34.399397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 577/1187] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:34.421869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e01a8-da39-44a7-b01f-845fb1065d72 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:34.521243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e01a8-da39-44a7-b01f-845fb1065d72 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:34.553648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:34.554419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:34.554701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:34.555007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:34.565272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:52:34.572709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:34.573618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:52:34.574469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 611/1188] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:34.579925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:34.677370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc2020-e027-414e-a0ce-c9f8033493a6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:34.701155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:34.713533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:34.714255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:34.714542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:34.714868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:34.715800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:34.716108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:34.717474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:34.726203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 578/1189] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:34.732095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:34.771856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:34.829655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a564d-1030-491e-801c-e1ab78c85315 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:34.862969 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:34.863834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:34.864118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:34.864396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] PATH_INFO: `/v3/users/9d4e11745b354abb81c487e0d7789a8c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:34.865575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] RBAC: Authorizing `identity:get_user(user_id=9d4e11745b354abb81c487e0d7789a8c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:34.867057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:34.868686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 612/1190] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:34 2022] GET /identity/v3/users/9d4e11745b354abb81c487e0d7789a8c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:34.874528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:34.972044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be378311-e584-4252-a4aa-612c8fb1caa8 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:35.005005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212f1c1-20a7-456a-abc9-685049b51f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:35.005859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212f1c1-20a7-456a-abc9-685049b51f14 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:35.006144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212f1c1-20a7-456a-abc9-685049b51f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:35.006421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212f1c1-20a7-456a-abc9-685049b51f14 None admin] PATH_INFO: `/v3/projects/c30a3afb7c804f759834d45e9bef334d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:35.007616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212f1c1-20a7-456a-abc9-685049b51f14 None admin] RBAC: Authorizing `identity:get_project(project_id=c30a3afb7c804f759834d45e9bef334d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:35.009047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212f1c1-20a7-456a-abc9-685049b51f14 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:35.010727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 579/1191] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:34 2022] GET /identity/v3/projects/c30a3afb7c804f759834d45e9bef334d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:35.016534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:35.115001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a05c18-47c3-46cb-8413-cc68eb5105ec None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:35.121519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:35.148424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb796d-e0d0-4fbc-b7e5-34bb699ced04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:35.149298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb796d-e0d0-4fbc-b7e5-34bb699ced04 None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:35.149580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb796d-e0d0-4fbc-b7e5-34bb699ced04 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:35.149854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb796d-e0d0-4fbc-b7e5-34bb699ced04 None admin] PATH_INFO: `/v3/projects/c30a3afb7c804f759834d45e9bef334d/users/9d4e11745b354abb81c487e0d7789a8c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:35.151804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb796d-e0d0-4fbc-b7e5-34bb699ced04 None admin] RBAC: Authorizing `identity:create_grant(project_id=c30a3afb7c804f759834d45e9bef334d, user_id=9d4e11745b354abb81c487e0d7789a8c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:35.153247 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb796d-e0d0-4fbc-b7e5-34bb699ced04 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:35.166079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 613/1192] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:52:35 2022] PUT /identity/v3/projects/c30a3afb7c804f759834d45e9bef334d/users/9d4e11745b354abb81c487e0d7789a8c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:52:37.367402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787331d-0ee8-44c8-a596-de2198352afe None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:37.367980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787331d-0ee8-44c8-a596-de2198352afe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:37.367980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787331d-0ee8-44c8-a596-de2198352afe None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:37.369344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 580/1193] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:52:37.374501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:37.374940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:37.375102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:37.391609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:52:37.393379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:52:37.458422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 614/1194] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:52:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:52:37.482077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787331d-0ee8-44c8-a596-de2198352afe None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:37.537509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:37.582330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787331d-0ee8-44c8-a596-de2198352afe None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:37.614651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:37.615448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:37.615737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:37.616015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:37.626313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:52:37.634214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:52:37.635334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:52:37.636215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 581/1195] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:52:37.641267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:37.732910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:37.740073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc31806-aef5-4273-88b1-ad472e330b5d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:37.774064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:37.774724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:37.775039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:37.775319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:37.776060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:37.776370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:37.777758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:37.785645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 615/1196] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:52:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:37.791302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:37.893930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6352f8b-b06b-4225-b781-dfd4317052bc None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:37.907710 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:37.927629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:37.928497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:37.928782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:37.929059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] PATH_INFO: `/v3/users/9d4e11745b354abb81c487e0d7789a8c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:37.930235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] RBAC: Authorizing `identity:get_user(user_id=9d4e11745b354abb81c487e0d7789a8c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:37.931686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:37.933299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 582/1197] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 11:52:37 2022] GET /identity/v3/users/9d4e11745b354abb81c487e0d7789a8c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:37.940024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:38.042280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e656-3560-488a-a15c-a7ced9464dd9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:38.075437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:38.076076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:38.076363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:38.076642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:38.102706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:38.143033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:38.144540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:38.240129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 616/1198] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:52:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:52:38.245945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:38.341968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec379a-c0ee-47d9-8f09-82f78746762d None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:38.375447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f876c0-916d-4f8b-80a1-6a8265c6019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:38.376294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f876c0-916d-4f8b-80a1-6a8265c6019e None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:38.376579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f876c0-916d-4f8b-80a1-6a8265c6019e None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:38.377412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f876c0-916d-4f8b-80a1-6a8265c6019e None admin] PATH_INFO: `/v3/projects/c30a3afb7c804f759834d45e9bef334d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:38.378734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f876c0-916d-4f8b-80a1-6a8265c6019e None admin] RBAC: Authorizing `identity:get_project(project_id=c30a3afb7c804f759834d45e9bef334d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:38.380195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f876c0-916d-4f8b-80a1-6a8265c6019e None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:38.381856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 583/1199] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 11:52:38 2022] GET /identity/v3/projects/c30a3afb7c804f759834d45e9bef334d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:38.387735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:52:38.443982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:38.484696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c3dd-4f89-4e6d-8006-40da82c00b3b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:52:38.517835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:52:38.518476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:52:38.518766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:52:38.519079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:52:38.519905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=c30a3afb7c804f759834d45e9bef334d, user.id=9d4e11745b354abb81c487e0d7789a8c) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:52:38.520222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:52:38.521666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a014ff-d0ea-4c40-a967-8b034044c6d9 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:52:38.527431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 617/1200] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:52:38 2022] GET /identity/v3/role_assignments?user.id=9d4e11745b354abb81c487e0d7789a8c&scope.project.id=c30a3afb7c804f759834d45e9bef334d&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:52:48.371292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=75 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:52:48.525259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=39 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:39.448981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a6e9c-cfba-42cc-8b3a-e8290bf6c849 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:39.449812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a6e9c-cfba-42cc-8b3a-e8290bf6c849 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:39.449812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a6e9c-cfba-42cc-8b3a-e8290bf6c849 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:39.451372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 584/1201] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:53:39.458014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:39.458624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:39.458883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:39.481179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:53:39.481179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:53:39.555691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 618/1202] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:53:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:53:39.584030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a6e9c-cfba-42cc-8b3a-e8290bf6c849 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:39.724653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a6e9c-cfba-42cc-8b3a-e8290bf6c849 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:39.764444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:39.765607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:39.765885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:39.766165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:39.781346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:39.781346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:39.803403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:39.804444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:39.805510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:39.805510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:39.805510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:39.805510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:39.807943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 585/1203] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:39.813757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:39.944894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100c02c-c0e7-4566-9297-d10c2a2866b2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:39.981616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:39.987864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:39.988900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:39.989382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:39.989659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:39.990471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:39.990706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:39.992272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ade5c-fa9c-432f-abc8-278738c63fef None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:40.005150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 619/1204] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:53:40.013949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:40.086380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:40.149717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c1f4c-8d6a-4bff-ac90-aac387905095 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:40.192038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df0f58-70ae-49b8-9261-d5540b58072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:40.193055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df0f58-70ae-49b8-9261-d5540b58072a None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:40.193350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df0f58-70ae-49b8-9261-d5540b58072a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:40.194303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df0f58-70ae-49b8-9261-d5540b58072a None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:40.194599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df0f58-70ae-49b8-9261-d5540b58072a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:40.196254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df0f58-70ae-49b8-9261-d5540b58072a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:40.221497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 586/1205] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 11:53:40 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:53:42.875534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f18df-b56b-475c-874d-77e60c2f3b0f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:42.876098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f18df-b56b-475c-874d-77e60c2f3b0f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:42.876098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f18df-b56b-475c-874d-77e60c2f3b0f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:42.877739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 620/1206] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:53:42.883783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:42.884288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:42.884405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:42.904913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:53:42.906430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:53:42.991226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 587/1207] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:53:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:53:43.016116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f18df-b56b-475c-874d-77e60c2f3b0f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:43.137691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f18df-b56b-475c-874d-77e60c2f3b0f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:43.181252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:43.182017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:43.182321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:43.182583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:43.194959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:53:43.202554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:43.203698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:53:43.204652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:43.209095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 621/1208] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:53:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:43.210769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:43.258941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:43.328594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567dc8d-4073-44b0-befc-57263464c6f8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:43.376931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:43.377732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:43.378025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:43.378349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:43.379419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:43.379846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:43.381385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:43.391174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 588/1209] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:53:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:43.398943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:43.459324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:43.518535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7be494-0835-47ad-b8f5-f9dacf0c5164 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:43.561719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:43.562522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:43.562806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:43.563127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:43.574555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:43.576159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:43.594434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:43.595491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:43.596428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:43.596428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:43.596428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:43.596428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:43.596428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 622/1210] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:43.605616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:43.721061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84043575-3e9b-4713-9d2e-f7a724b81404 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:43.723401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:43.768402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:43.769508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:43.770375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:43.771186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:43.772409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:43.772765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:43.774590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:43.797415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 589/1211] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:53:43.798719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:43.923812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82444625-ec39-43e3-baed-9c294ff970fd None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:43.925714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:43.966021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:43.967019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:43.967324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:43.967696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:43.976820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:43.978237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:53:43.999133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:44.000236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:44.001202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:44.007073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 623/1212] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:44.009417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:44.172060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938e1b3-f3d4-45bd-9ee6-6d29391fa494 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:44.221796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:44.222600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:44.223215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:44.223606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:44.224509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:44.224952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:44.226499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:44.249011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 590/1213] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 11:53:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:44.253853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:44.377522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da48c90-9558-4fbd-a4f2-058b55207e20 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:44.424071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:44.425042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:44.425487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:44.426451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:44.463693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:44.518639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:44.520807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:44.641970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 624/1214] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:53:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:53:44.643341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:44.670082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:44.781061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316b61b-f33c-4952-b863-46d4b99162b5 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:44.818222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:44.819050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:44.819338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:44.819630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:44.831899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:44.834212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:44.856772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:44.857810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:44.858954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:44.858954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:44.858954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:44.858954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:44.858954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 591/1215] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:44.864695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:44.937008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:44.982750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3186-9a49-46c8-b06d-2331c45775eb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:45.023996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:45.024877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:45.025206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:45.025558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:45.026432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:45.026744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:45.028419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:45.043099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 625/1216] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:45.049097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:45.132916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:45.171592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea557c9d-0c3b-47fb-89b4-0c258ce31bec None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:45.213402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:45.214334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:45.214759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:45.215151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:45.223978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:45.225562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:53:45.239520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:45.240675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:45.241861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:45.248186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 592/1217] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:53:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:45.249396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:45.383038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d9d98-063e-42a9-95de-f3a0b25a7685 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:45.421412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:45.437219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:45.437885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:45.438172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:45.438452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:45.439270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:45.439583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:45.441034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b4f0-77ae-4daa-a38a-c2dbe96f4b20 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:45.456464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 626/1218] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:53:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:53:45.463219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:45.592880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb98b3-6b79-4958-9740-b3ef0fce6bd9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:45.628013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:45.636150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:45.636794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:45.637103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:45.637474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:45.638452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=9c11a2d80dcf4533a2b87fb5d85a552c) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:45.638885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:45.640752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f676b4-0c65-45b9-87a5-e73d65934ef5 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:45.649165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 593/1219] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:53:45 2022] GET /identity/v3/role_assignments?user.id=9c11a2d80dcf4533a2b87fb5d85a552c&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:48.095007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ec8a0-2b2d-4988-96b5-6d8f2173d2b6 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:48.095527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ec8a0-2b2d-4988-96b5-6d8f2173d2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:48.095639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ec8a0-2b2d-4988-96b5-6d8f2173d2b6 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:48.096983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 627/1220] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:53:48.105618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:48.105618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:48.105618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:48.122725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:53:48.124217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:53:48.204988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 594/1221] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:53:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:53:48.229713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ec8a0-2b2d-4988-96b5-6d8f2173d2b6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:48.354690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ec8a0-2b2d-4988-96b5-6d8f2173d2b6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:48.391358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:48.392153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:48.392439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:48.392718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:48.403201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:53:48.412574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:48.413614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:48.414463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 628/1222] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:53:48.420109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:48.525528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd7efa-1532-4421-9530-80bd59e83b93 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:48.549168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:48.561415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:48.562133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:48.562472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:48.562855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:48.563620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:48.563936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:48.565307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:48.575310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 595/1223] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:53:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:48.581242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:48.625187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:48.688104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343f07b-6f19-4d16-adbf-36fe4adda2b9 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:48.729065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:48.729957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:48.730242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:48.730554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:48.744248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:48.746002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:48.767118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:48.768379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:48.769474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:48.769474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:48.769474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:48.769474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:48.769474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 629/1224] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:48.776049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:48.888213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a87b17-efbc-40b2-99a0-7fcb71ea1f91 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:48.924942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:48.925730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:48.926170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:48.926988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:48.927912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:48.928285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:48.929722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:48.946871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 596/1225] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:48.951896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:49.062297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:49.069841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472186e2-d817-4f22-b35f-14effc5f1883 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:49.110291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:49.111175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:49.111465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:49.111746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:49.121160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:49.122871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:53:49.138584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:49.139815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:49.140839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 630/1226] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:49.145953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:49.198535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:49.256246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49711-6d15-4c6e-bdfb-d732635e2f87 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:49.291111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:49.291790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:49.292076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:49.292358 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:49.293193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:49.293507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:49.294968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:49.314273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 597/1227] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 11:53:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:49.321027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:49.423048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46813b04-bdf2-4055-a5ad-a30ad0c327d6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:49.459961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:49.460882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:49.461264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:49.461590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:49.473915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:49.475665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:49.496689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:49.497768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:49.498698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:49.498698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:49.498698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:49.498698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:49.498698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 631/1228] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:49.506779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:49.623912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1ed9e-e11e-49fa-afa1-0a7709a57bfd None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:49.626786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:49.667164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:49.667822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:49.668107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:49.668391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:49.669109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:49.669417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:49.670929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:49.684012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 598/1229] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:49.689367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:49.699693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:49.811480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be4798-d6ed-4ea8-b060-bfadba497052 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:49.849842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:49.850747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:49.851293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:49.851545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:49.858737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:49.860277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:53:49.875748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:49.877652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:49.879173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 632/1230] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:49.884047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:49.991406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e21d3-1b0e-45c1-b485-8f249ed850f3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:50.028601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:50.029279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:50.029633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:50.029956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:50.030821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:50.031224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:50.032849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299e8a1-07cd-43f9-b223-ac44ae783df2 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:50.049160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 599/1231] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:50.056313 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:50.130631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:50.169422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae14523-c259-4c66-a3c4-e7107af0832e None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:50.204592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9475f9-b674-46f8-846f-dba9678c463e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:50.205590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9475f9-b674-46f8-846f-dba9678c463e None admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:50.205873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9475f9-b674-46f8-846f-dba9678c463e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:50.206145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9475f9-b674-46f8-846f-dba9678c463e None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/9c11a2d80dcf4533a2b87fb5d85a552c/roles/a01c521c59bf4a59968f9bff26d7c7ab` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:50.224595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9475f9-b674-46f8-846f-dba9678c463e None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=9c11a2d80dcf4533a2b87fb5d85a552c, role_id=a01c521c59bf4a59968f9bff26d7c7ab)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:50.226226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9475f9-b674-46f8-846f-dba9678c463e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:50.240718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 633/1232] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:53:50 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/9c11a2d80dcf4533a2b87fb5d85a552c/roles/a01c521c59bf4a59968f9bff26d7c7ab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:53:52.733513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cedaf-c627-4de3-b57d-eab7334b6341 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:52.734121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cedaf-c627-4de3-b57d-eab7334b6341 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:52.734121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cedaf-c627-4de3-b57d-eab7334b6341 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:52.735625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 600/1233] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:53:52.744670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:52.744670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:52.744670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:52.766277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:53:52.767742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:53:52.866874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 634/1234] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:53:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:53:52.893375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cedaf-c627-4de3-b57d-eab7334b6341 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:52.925578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:53.023714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cedaf-c627-4de3-b57d-eab7334b6341 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:53.064325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:53.065202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:53.065494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:53.065792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:53.077537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:53:53.085354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:53.086446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:53.087419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 601/1235] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 11:53:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:53.092448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:53.142915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:53.201883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9b10f-bc1e-44e6-8ad6-abb5be8fee8a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:53.237806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:53.238489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:53.239354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:53.239657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:53.240412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:53.240771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:53.242383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:53.252257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 635/1236] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 11:53:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:53.258814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:53.349377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:53.368179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4de129-1081-49c8-9f45-64151b3f16ae None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:53.405792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:53.406717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:53.407087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:53.407406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:53.419796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:53.421277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:53.439945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:53.441048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:53.442587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:53.442587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:53.442587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:53.442587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:53.442587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 602/1237] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:53.448163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:53.555376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc62d9-0bc1-40aa-8955-d58440a93d4d None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:53.565929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:53.593033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:53.593746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:53.594063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:53.594345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:53.595143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:53.595460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:53.596981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:53.613348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 636/1238] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:53.619869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:53.727938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f1856-5e78-4e9f-a735-72f497311337 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:53.774426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:53.775336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:53.775668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:53.776032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:53.779652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:53.786737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:53.788280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:53:53.807273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:53.808687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:53.809955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 603/1239] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:53.814529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:53.922963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30894ba-56cb-4be3-bd7c-39c51d64e0a2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:53.958416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:53.959324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:53.959709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:53.959998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:53.960839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:53.961152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:53.962603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:53.981341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 637/1240] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 11:53:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:53.987298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:54.098936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3e55-4f7c-4b5a-85ab-46636dfae467 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:54.135491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:54.136146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:54.136436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:54.136717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:54.204913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:54.212289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:54.213826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:54.318711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 604/1241] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:53:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 11:53:54.324746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:54.354069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:54.435010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f90a8-cf06-4179-bfbd-a9be6cf5df15 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:54.471083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:54.472234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:54.472545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:54.472848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:54.485617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:54.487299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:54.507391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:54.508832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:54.509956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:54.509956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:54.509956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:54.509956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:54.509956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 638/1242] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:54.515424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:54.619727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:54.627017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2068438-eaf2-4053-8fb1-eeb41fd2e9e9 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:54.663259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:54.664059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:54.664390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:54.664674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:54.665408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:54.665716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:54.667133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:54.681158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 605/1243] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:53:54.686829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:54.775999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:54.803722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacfd3d-e2c9-488c-9fa3-0d309e8f0e86 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:54.841860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:54.842922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:54.843276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:54.843612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:54.853367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:54.854858 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:53:54.872196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:54.873418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:54.878990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 639/1244] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:53:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:53:54.880256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:55.002481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98355f6-8d5d-4027-a27c-b60c5d2812fe None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:55.041844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:55.043210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:55.043544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:55.043849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:55.044642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:55.044959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:55.046932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cf0c5-e7e8-46f4-98ac-536bbd03bc95 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:55.064336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 606/1245] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:53:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:55.070447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:55.181781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a505-818f-4ba0-9fa8-34ca6d9907de None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:55.218274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:55.218911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:55.219200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:55.219484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:55.220324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] RBAC: Adding query filter params (role.id=a01c521c59bf4a59968f9bff26d7c7ab, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=9c11a2d80dcf4533a2b87fb5d85a552c) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:55.220641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:55.222090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebde16-0598-4e5c-af26-96b2fc32747e None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:55.228691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 640/1246] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:53:55 2022] GET /identity/v3/role_assignments?user.id=9c11a2d80dcf4533a2b87fb5d85a552c&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=a01c521c59bf4a59968f9bff26d7c7ab => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 11:53:57.748073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68759f0b-db37-4e23-8bd7-80a306b4ac30 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:57.748610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68759f0b-db37-4e23-8bd7-80a306b4ac30 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:57.748728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68759f0b-db37-4e23-8bd7-80a306b4ac30 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:57.750015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 607/1247] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:53:57.756860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:57.757293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:57.757412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:57.762691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:57.778354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:53:57.780013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:53:57.859678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 641/1248] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:53:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 11:53:57.883969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68759f0b-db37-4e23-8bd7-80a306b4ac30 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:57.904025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:58.000080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68759f0b-db37-4e23-8bd7-80a306b4ac30 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:58.046258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:58.047293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:58.047681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:58.048028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:58.059874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:53:58.069370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:58.070546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:58.071680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 608/1249] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:58.076926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:58.196091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2d2fe-f5ba-432f-8fdc-8764ae1eb538 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:58.229056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:58.234613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:58.235336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:58.235628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:58.235913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:58.236663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:58.236976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:58.238364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:58.252884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 642/1250] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:53:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:58.254027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:58.342740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:58.372408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39593eb1-fc17-451b-a232-bf1c3d8f9b90 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:58.410091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:58.410913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:58.411460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:58.411875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:58.422372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:58.423868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:58.443159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:58.444133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:58.445129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:58.445129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:58.445129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:58.445129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:58.445129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 609/1251] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:53:58.450140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:58.570877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a1600-58bd-4d34-92b8-4cad65e4c18c None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:58.609961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:58.610627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:58.610952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:58.611264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:58.612069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:58.612432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:58.613919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:58.627879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 643/1252] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:58.633900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:58.746750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a7ac-d7a1-4fe7-8646-e7ce3788f4ad None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:58.775868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:58.788893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:58.790034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:58.790421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:58.790897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:58.801410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:58.803955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:53:58.821408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:58.822513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:53:58.823644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 610/1253] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:58.828808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:58.890720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:58.937656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75590b3e-d635-47e8-9efe-d08fb0eaf411 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:58.973813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:58.974741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:58.975065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:58.975322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:58.976185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:58.976475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:58.977947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:58.997958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 644/1254] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 11:53:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:59.004007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:59.123556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4930b-dc2c-48cc-b29c-d45760954f2b None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:59.160050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:59.160726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:59.161056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:59.161937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:59.220675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:59.234667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:59.236185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:59.344922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 611/1255] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:53:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 11:53:59.346127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:59.452006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69eeaa-129c-4fac-a06e-fc38294dbd08 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:59.466312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:59.490466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:59.491317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:59.491658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:59.491942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:59.502951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:59.504446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:53:59.523008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:59.524060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:59.525117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:53:59.525117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:53:59.525117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:53:59.525117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:59.525117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 645/1256] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:59.530139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:59.624498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:53:59.637627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327d81b-2061-4611-8ebf-b6c2e80d6c8f None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:59.673883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:59.674558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:59.674910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:59.675147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:59.675867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:53:59.676179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:59.678323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:59.689974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 612/1257] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:53:59.695814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:59.813939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97370d4a-67ee-49f3-aad7-a29d47a1f7de None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:53:59.847182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:53:59.847968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:53:59.848250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:53:59.848635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:53:59.856268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:53:59.857928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:53:59.872368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:53:59.873439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:53:59.874477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 646/1258] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:53:59.879391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:53:59.976753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324e3a2-b2e6-4588-bbc3-bd0f51775481 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:00.009056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:00.009714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:00.010000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:00.010281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:00.011094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:00.011414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:00.012843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cb4c6-9e39-4897-a6d1-1c283909b625 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:00.025319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 613/1259] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:00.031224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:00.056664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:00.129583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4394c-a83a-4119-a0c9-698298df9089 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:00.162436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:00.163125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:00.163419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:00.163705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:00.164540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=9c11a2d80dcf4533a2b87fb5d85a552c) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:00.164862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:00.166318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add276bd-260f-46c9-b469-1fbcadb37eae None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:00.171962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 647/1260] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:54:00 2022] GET /identity/v3/role_assignments?user.id=9c11a2d80dcf4533a2b87fb5d85a552c&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=b4418dc4009641b183fe42e56f1e484e => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:02.391174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6289df-030c-44fe-8611-e6f669330a57 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:02.391692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6289df-030c-44fe-8611-e6f669330a57 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:02.391811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6289df-030c-44fe-8611-e6f669330a57 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:02.393110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 614/1261] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:54:02.398786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:02.399251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:02.399389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:02.415979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:54:02.417201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:54:02.483224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 648/1262] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:54:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:54:02.506463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6289df-030c-44fe-8611-e6f669330a57 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:02.517194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:02.608030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6289df-030c-44fe-8611-e6f669330a57 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:02.640683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:02.641514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:02.641745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:02.642025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:02.652420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:54:02.659913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:02.660998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:02.662003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 615/1263] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:54:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:02.667124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:02.684914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:02.769234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12f1dc-e62c-4e08-a404-20574c9da66b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:02.807924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:02.808650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:02.808937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:02.809222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:02.809968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:02.810336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:02.811696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:02.820823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 649/1264] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:54:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:02.826580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:02.897417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:02.931238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03463e-6cd7-4fa8-adf8-fd50afc64671 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:02.964108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:02.964897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:02.965190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:02.965473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:02.976068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:02.977513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:54:02.994587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:02.995643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:02.996566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:54:02.996566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:54:02.996566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:02.996566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:02.996566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 616/1265] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:03.001529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:03.080784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:03.101025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f9718-3690-4711-98b1-8c7292763744 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:03.133789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:03.134435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:03.134718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:03.135024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:03.135731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:03.136036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:03.137404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:03.148883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 650/1266] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:03.154558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:03.253750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda859-d909-4330-b7ec-6a74c70fba3a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:03.273053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:03.287448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:03.288243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:03.288528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:03.288805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:03.296964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:03.298396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:54:03.312944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:03.314043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:03.315252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 617/1267] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:03.320050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:03.417551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea058a-3ce0-4666-a1e4-58b2dbcecafb None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:03.450540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:03.451235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:03.451527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:03.451812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:03.452638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:03.452979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:03.454422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:03.463407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:03.472083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 651/1268] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:03.478553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:03.573092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994b964-0e48-4884-9983-1c0929c566f0 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:03.606066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:03.606861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:03.607147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:03.607424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:03.617685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:03.619149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:54:03.635510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:03.636565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:03.642261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:54:03.642261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:54:03.642261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:03.642261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:03.642261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 618/1269] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:03.642624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:03.741828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daef49-3645-41ab-903d-cbf6e00217e6 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:03.782222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:03.782942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:03.783227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:03.783501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:03.784213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:03.784518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:03.785916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:03.797958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 652/1270] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:03.803892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:03.835567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:03.902522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5849efe-a7e3-40e7-a8da-b72a44184e3a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:03.934360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:03.935174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:03.935458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:03.935734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:03.942207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:03.944183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:54:03.957908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:03.959673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:03.960868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 619/1271] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 11:54:03.965582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:04.024592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:04.065515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce33ed-9c0f-4757-9cbc-bf4c41081ec7 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:04.098890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:04.099847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:04.100141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:04.100422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:04.101197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:04.101509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:04.102978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0806c6-c673-4bac-8835-259da97c37c7 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:04.115502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 653/1272] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:54:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:04.121529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:04.210224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:04.219244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44498d5-b41d-44a4-94f2-ad98a04aaf74 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:04.252985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be69de-3492-4cc9-a893-91e82d522f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:04.253866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be69de-3492-4cc9-a893-91e82d522f89 None admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:04.254149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be69de-3492-4cc9-a893-91e82d522f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:04.254423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be69de-3492-4cc9-a893-91e82d522f89 None admin] PATH_INFO: `/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/9c11a2d80dcf4533a2b87fb5d85a552c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:04.257119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be69de-3492-4cc9-a893-91e82d522f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf85d790611946309fcb27d7ad7aef17, user_id=9c11a2d80dcf4533a2b87fb5d85a552c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:04.258554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be69de-3492-4cc9-a893-91e82d522f89 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:04.271052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 620/1273] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 11:54:04 2022] PUT /identity/v3/projects/cf85d790611946309fcb27d7ad7aef17/users/9c11a2d80dcf4533a2b87fb5d85a552c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 11:54:06.485718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683657-947a-4652-9953-c2aa14dc0d02 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:06.486245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683657-947a-4652-9953-c2aa14dc0d02 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:06.486333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63683657-947a-4652-9953-c2aa14dc0d02 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:06.487681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 654/1274] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:54:06.493310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:06.493731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:06.493875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:06.510729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:54:06.511956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:54:06.575551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 621/1275] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:54:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:54:06.598953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683657-947a-4652-9953-c2aa14dc0d02 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:06.699932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63683657-947a-4652-9953-c2aa14dc0d02 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:06.716901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:06.734036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:06.734819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:06.735134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:06.735413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:06.747669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 11:54:06.755553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:06.756804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:06.757895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 655/1276] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 11:54:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:06.763428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:06.850513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:06.869575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0e25a-2eb3-4069-bd2b-fa32bb73e411 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:06.903251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:06.903917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:06.904204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:06.904483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:06.905219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:06.905524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:06.906977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:06.915554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 622/1277] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 11:54:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:06.921183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:07.020186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a40bf2-14bb-42e7-b30f-041bd8af2aaf None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:07.053769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:07.054549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:07.054834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:07.055140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:07.065280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:07.066747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:54:07.083209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:07.084282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:07.085520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:54:07.085520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:54:07.085520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:07.085520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:07.085520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 656/1278] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:07.090052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:07.191911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d20e0-46db-4d18-b8b8-a161d3f9e022 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:07.226930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:07.227664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:07.227957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:07.228279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:07.228995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:07.229328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:07.230885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:07.242976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 623/1279] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:07.248398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:07.279949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:07.348647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de8321-3714-47d7-ac3a-54d760bf6e61 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:07.382713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:07.383525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:07.383806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:07.384082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:07.391613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:07.393066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 11:54:07.407115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:07.408219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:07.409239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 657/1280] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:54:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:07.414374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:07.418314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:07.512168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970a53f-dc4d-481a-a14d-07573b5709b1 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:07.544862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:07.545520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:07.545808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:07.546089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:07.546941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:07.547256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:07.548681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:07.565733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 624/1281] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 11:54:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:07.572262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:07.660811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:07.671183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af84a1-981b-4e89-b262-a73f9e0e9581 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:07.704161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:07.704795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:07.705085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:07.705365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:07.772914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:07.774396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:07.860960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:07.871962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 658/1282] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 11:54:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 11:54:07.877751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:07.942534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:07.980830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947b643-3ef9-4356-b025-8ee4b02b7d39 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:08.014310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:08.015134 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:08.015432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:08.015691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:08.026249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:08.027718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 11:54:08.045128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:08.046238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:08.047352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 11:54:08.047352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 11:54:08.047352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 11:54:08.047352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:08.047352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 625/1283] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:08.052051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:08.148367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5995e6-a6f5-4632-9546-7ae09d219937 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:08.181603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:08.182264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:08.182548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:08.182830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:08.183907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:08.184458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:08.186118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:08.197934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 659/1284] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 11:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:08.203576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:08.301190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021262d-7c1b-4649-945e-a1f0b7797bff None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:08.314800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:08.335599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:08.336404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:08.336677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:08.336958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:08.343452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:08.344911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 11:54:08.358729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:08.359777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: ERROR keystone.server.flask.application Jun 06 11:54:08.360917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 626/1285] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 11:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:08.365825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:08.415209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:08.469193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5b96-d62b-4a70-bea7-23ed1bb6a7a2 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:08.501378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:08.502029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:08.502316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:08.502597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:08.503404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:08.503718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:08.505163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bf918-536a-4bf9-b691-654dd26e3292 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:08.517308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 660/1286] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 11:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:08.524104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:08.621294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5bd8b-aaa0-40c0-9cf6-b831fbca3dd3 None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:08.655248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:08.655885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:08.656169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:08.656442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:08.657251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] RBAC: Adding query filter params (role.id=b4418dc4009641b183fe42e56f1e484e, scope.project.id=cf85d790611946309fcb27d7ad7aef17, user.id=9c11a2d80dcf4533a2b87fb5d85a552c) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:08.657560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:08.659001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49506dd3-c4e4-4163-9e4b-895266ad176d None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:08.664778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 627/1287] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 11:54:08 2022] GET /identity/v3/role_assignments?user.id=9c11a2d80dcf4533a2b87fb5d85a552c&scope.project.id=cf85d790611946309fcb27d7ad7aef17&role.id=b4418dc4009641b183fe42e56f1e484e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:10.917086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba8669-bce8-497f-8e5e-fb9d802f07f3 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:10.917646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba8669-bce8-497f-8e5e-fb9d802f07f3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:10.917646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba8669-bce8-497f-8e5e-fb9d802f07f3 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:10.919185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 661/1288] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:54:10.924522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:10.924941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:10.925087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:10.942390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:54:10.943833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:54:10.972399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:11.008960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 628/1289] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:54:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:54:11.031446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba8669-bce8-497f-8e5e-fb9d802f07f3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:11.131834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba8669-bce8-497f-8e5e-fb9d802f07f3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:11.142035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:11.165994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:11.166787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:11.167152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:11.167382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:11.168026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:11.169404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:11.181003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:11.182202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:11.183477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 662/1290] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:54:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:11.188229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:11.283599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b39d18-e4ad-4fd9-bdee-ab7ec3d4b879 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:11.316938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:11.317587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:11.318399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:11.318988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:11.319455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:11.319763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:11.321120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9483-97b3-46f1-aebf-6c36d474880a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:11.325813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 629/1291] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 11:54:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:11.331389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:11.428966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6040035-cfce-4f14-8a59-c130641c4502 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:11.462401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:11.463082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:11.463369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:11.463650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] PATH_INFO: `/v3/services` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:11.464375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:11.464686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:11.466056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b66237-b21c-432e-87ec-b8c169d9b893 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:11.470451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 663/1292] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 11:54:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:13.707504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b7b3-f0c5-4033-bf72-2e7180e0d3eb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:13.708061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b7b3-f0c5-4033-bf72-2e7180e0d3eb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:13.708061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b7b3-f0c5-4033-bf72-2e7180e0d3eb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:13.709806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 630/1293] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:54:13.715095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47113179-0c92-4c6c-b4d2-229dcb420c4c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:13.715461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47113179-0c92-4c6c-b4d2-229dcb420c4c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:13.715655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47113179-0c92-4c6c-b4d2-229dcb420c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:13.733332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-47113179-0c92-4c6c-b4d2-229dcb420c4c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:54:13.734508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-47113179-0c92-4c6c-b4d2-229dcb420c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:54:13.768818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:13.811717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 664/1294] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:54:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:54:13.836858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b7b3-f0c5-4033-bf72-2e7180e0d3eb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:13.868759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:13.937611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b7b3-f0c5-4033-bf72-2e7180e0d3eb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:13.970637 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e896c78-fd60-4d83-a909-61cda5b8d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:13.971256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896c78-fd60-4d83-a909-61cda5b8d0dc None admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:13.971543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896c78-fd60-4d83-a909-61cda5b8d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:13.971824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e896c78-fd60-4d83-a909-61cda5b8d0dc None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:13.972532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e896c78-fd60-4d83-a909-61cda5b8d0dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:13.973890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e896c78-fd60-4d83-a909-61cda5b8d0dc None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:13.986869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 631/1295] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 11:54:13 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:54:16.190679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa98fda-8799-4d76-84a3-80b46602ed4d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:16.191350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa98fda-8799-4d76-84a3-80b46602ed4d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:16.191350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa98fda-8799-4d76-84a3-80b46602ed4d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:16.192683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 665/1296] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:54:16.198300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:16.198698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:16.198887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:16.216134 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:54:16.217306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:54:16.288267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 632/1297] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:54:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:54:16.310806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa98fda-8799-4d76-84a3-80b46602ed4d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:16.409151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa98fda-8799-4d76-84a3-80b46602ed4d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:16.442105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:16.442917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:16.443830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:16.444209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:16.444940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:16.446337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:16.457714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:16.458941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:16.460014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 666/1298] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:54:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:16.464932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:16.483032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:16.566275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85943e-2cde-4c5b-9894-f3c7545b2b3c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:16.599130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:16.600458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:16.600997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:16.601272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:16.601968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:16.602273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:16.603653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:16.609345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 633/1299] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 11:54:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:16.615935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:16.648533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:16.719853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d342900-85ed-4a4f-9d16-4a4aff72542b None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:16.757241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:16.757900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:16.758187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:16.758468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:16.759301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] RBAC: Adding query filter params (interface=public, service_id=a77d809cb5814f1ea43768adc8d80d60, region_id=RegionOne) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:16.759862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:16.761279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf678d-7de6-4b51-bafb-af1fd7025f30 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:16.766432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 667/1300] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 11:54:16 2022] GET /identity/v3/endpoints?service_id=a77d809cb5814f1ea43768adc8d80d60&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:16.772145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:16.868786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:16.892143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d152f-1545-4269-8633-21dd5dff6a5a None admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:16.928930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd1ea-2f2c-45c4-8b0d-594466b484bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:16.929792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1ea-2f2c-45c4-8b0d-594466b484bb None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:16.930079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1ea-2f2c-45c4-8b0d-594466b484bb None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:16.930355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1ea-2f2c-45c4-8b0d-594466b484bb None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:16.931053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abd1ea-2f2c-45c4-8b0d-594466b484bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:16.932434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abd1ea-2f2c-45c4-8b0d-594466b484bb None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:16.939690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 634/1301] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 11:54:16 2022] GET /identity/v3/services => generated 2191 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 11:54:19.212086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b8f48-e6df-4b71-9c08-95224d9ad89e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:19.212611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b8f48-e6df-4b71-9c08-95224d9ad89e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:19.212729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b8f48-e6df-4b71-9c08-95224d9ad89e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:19.214020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 668/1302] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:54:19.219145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:19.219516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:19.219727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:19.236329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:54:19.237521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:54:19.302287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 635/1303] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:54:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 11:54:19.325662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b8f48-e6df-4b71-9c08-95224d9ad89e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:19.419371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:19.426149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b8f48-e6df-4b71-9c08-95224d9ad89e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:19.458295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:19.459641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:19.459929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:19.460244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:19.460903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:19.462301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 11:54:19.473936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 11:54:19.475656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application with Lock( Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self._enter() Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application return self.creator() Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application created_value = creator( Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 11:54:19.476951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: ERROR keystone.server.flask.application Jun 06 11:54:19.478997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 669/1304] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 11:54:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 11:54:19.481991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:19.556058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:19.578375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c836aea-011d-464f-8dab-376e283f3821 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:19.610958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:19.611613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:19.611896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:19.612172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] PATH_INFO: `/v3/services` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:19.612860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 11:54:19.613166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:19.614524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825e14-0963-416f-91c8-e77cfa587e0c None admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:19.619519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 636/1305] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 11:54:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 11:54:19.625284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:54:19.724006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1804c-706d-4bae-84d4-e711b1b04585 None admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:54:19.757094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92d3c-bb29-47b0-b193-9ac0d41ac61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:54:19.757738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92d3c-bb29-47b0-b193-9ac0d41ac61a None admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:54:19.758025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92d3c-bb29-47b0-b193-9ac0d41ac61a None admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:54:19.758307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92d3c-bb29-47b0-b193-9ac0d41ac61a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:54:19.759077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae92d3c-bb29-47b0-b193-9ac0d41ac61a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:54:19.760453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae92d3c-bb29-47b0-b193-9ac0d41ac61a None admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:54:19.779227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 670/1306] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 11:54:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 11:54:29.616788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=30 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:54:29.774888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:23.744529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c527c-4364-496f-aa55-b5699b11e869 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:23.745591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c527c-4364-496f-aa55-b5699b11e869 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:23.745591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c527c-4364-496f-aa55-b5699b11e869 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:23.752649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21c97a-bc6f-4594-916d-fa20b74143c7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:23.753106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21c97a-bc6f-4594-916d-fa20b74143c7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:23.753198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21c97a-bc6f-4594-916d-fa20b74143c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:23.756877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 637/1307] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:23 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:23.826963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3a21c97a-bc6f-4594-916d-fa20b74143c7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:23.828167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3a21c97a-bc6f-4594-916d-fa20b74143c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:23.940127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 671/1308] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:23.992341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55034347-2800-43ee-b8e8-03cb1f63b6ad None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:23.992625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55034347-2800-43ee-b8e8-03cb1f63b6ad None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:23.992943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55034347-2800-43ee-b8e8-03cb1f63b6ad None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:23.994483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 638/1309] 10.222.0.41 () {54 vars in 964 bytes} [Mon Jun 6 11:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:24.002074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a0fec-4a1a-4a85-8749-5737945f0333 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:24.008378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a0fec-4a1a-4a85-8749-5737945f0333 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:24.008966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a0fec-4a1a-4a85-8749-5737945f0333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:24.168879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d59a0fec-4a1a-4a85-8749-5737945f0333 None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:24.171758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d59a0fec-4a1a-4a85-8749-5737945f0333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:24.228860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:24.329273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 672/1310] 10.222.0.41 () {58 vars in 1061 bytes} [Mon Jun 6 11:55:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:24.335854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55034347-2800-43ee-b8e8-03cb1f63b6ad None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:24.337659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-55034347-2800-43ee-b8e8-03cb1f63b6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:24.458235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55034347-2800-43ee-b8e8-03cb1f63b6ad None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:24.535900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:24.536463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:24.536747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:24.537021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:24.538642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:24.597223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:24.663957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:24.665443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d9385-ded5-4ad8-aa4d-af9ad0baa11d service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:24.690263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 639/1311] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:27.254770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce8ee-a519-4be7-b0d2-1d995697d7c3 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:27.255337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce8ee-a519-4be7-b0d2-1d995697d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:27.255428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce8ee-a519-4be7-b0d2-1d995697d7c3 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:27.256910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 673/1312] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:27.262110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe5a3a-24de-42d5-9742-776bc55ddc36 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:27.262558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe5a3a-24de-42d5-9742-776bc55ddc36 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:27.262653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe5a3a-24de-42d5-9742-776bc55ddc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:27.354975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-16fe5a3a-24de-42d5-9742-776bc55ddc36 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:27.356457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-16fe5a3a-24de-42d5-9742-776bc55ddc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:27.467575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 640/1313] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:27.506611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce8ee-a519-4be7-b0d2-1d995697d7c3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:27.508231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ff1ce8ee-a519-4be7-b0d2-1d995697d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:27.632373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce8ee-a519-4be7-b0d2-1d995697d7c3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:27.709771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:27.713206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:27.714000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:27.714667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:27.715390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:27.717336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:27.839372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:27.841514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81223c3-1b35-4634-aff8-edb2e0686eaf service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:27.864251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 674/1314] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:30.466754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acb46d-aacb-4a42-916e-8bb26f1ca6ed None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:30.467337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acb46d-aacb-4a42-916e-8bb26f1ca6ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:30.467337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acb46d-aacb-4a42-916e-8bb26f1ca6ed None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:30.468715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 641/1315] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:30.473927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d751f9-cbc2-45d8-a3ed-63f5d89aeaad None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:30.474371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d751f9-cbc2-45d8-a3ed-63f5d89aeaad None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:30.474477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d751f9-cbc2-45d8-a3ed-63f5d89aeaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:30.537569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-35d751f9-cbc2-45d8-a3ed-63f5d89aeaad None None] MFA Rules not processed for user `350c85611b984015a23d8c9c7772914f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:30.538757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-35d751f9-cbc2-45d8-a3ed-63f5d89aeaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:30.590033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:30.635694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 675/1316] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:30.675861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acb46d-aacb-4a42-916e-8bb26f1ca6ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:30.677429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a9acb46d-aacb-4a42-916e-8bb26f1ca6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:30.687932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:30.781092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acb46d-aacb-4a42-916e-8bb26f1ca6ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:30.851761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:30.852413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:30.852701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:30.852983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:30.855271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:30.919492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:30.966420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:30.967907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbcb8dd-1568-4c12-9235-5a67d3ce2fda service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:30.986491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 642/1317] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:33.805808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9af6-deae-4f6f-8336-c75867ce060d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:33.806337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9af6-deae-4f6f-8336-c75867ce060d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:33.806429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9af6-deae-4f6f-8336-c75867ce060d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:33.807834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 676/1318] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:33.813292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07a30e-2299-496c-b4f6-8b5bc07d1806 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:33.813715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07a30e-2299-496c-b4f6-8b5bc07d1806 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:33.813856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07a30e-2299-496c-b4f6-8b5bc07d1806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:33.877132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3b07a30e-2299-496c-b4f6-8b5bc07d1806 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:33.878333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3b07a30e-2299-496c-b4f6-8b5bc07d1806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:33.972432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 643/1319] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:34.011044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e9af6-deae-4f6f-8336-c75867ce060d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:34.013233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8a9e9af6-deae-4f6f-8336-c75867ce060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:34.115868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e9af6-deae-4f6f-8336-c75867ce060d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:34.188016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:34.188650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:34.188935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:34.189212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:34.190832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:34.198077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:34.299540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:34.301026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b18f6-3ba9-479a-b972-d12c02dc260a service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:34.319765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 677/1320] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:37.356642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b88e02-df42-4a0c-acfd-917a8dbe7bd3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:37.357237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b88e02-df42-4a0c-acfd-917a8dbe7bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:37.357237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b88e02-df42-4a0c-acfd-917a8dbe7bd3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:37.358599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 644/1321] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:37.364143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b0e5-232a-4aa0-b32a-9d594016e300 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:37.364555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b0e5-232a-4aa0-b32a-9d594016e300 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:37.364827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b0e5-232a-4aa0-b32a-9d594016e300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:37.430755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5008b0e5-232a-4aa0-b32a-9d594016e300 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:37.431973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5008b0e5-232a-4aa0-b32a-9d594016e300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:37.490013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:37.530822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 678/1322] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 11:55:37.566848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b88e02-df42-4a0c-acfd-917a8dbe7bd3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:37.568454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b8b88e02-df42-4a0c-acfd-917a8dbe7bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:37.590378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:37.676846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b88e02-df42-4a0c-acfd-917a8dbe7bd3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:37.753488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:37.754226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:37.754543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:37.755005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:37.756663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:37.828749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:37.866586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:37.868065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1f7ec-7fb4-4c88-8866-9dc175625207 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:37.888264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 645/1323] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:40.204480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644e117-6a9c-4a02-b08f-6b7d604dedc9 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:40.205035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644e117-6a9c-4a02-b08f-6b7d604dedc9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:40.205035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644e117-6a9c-4a02-b08f-6b7d604dedc9 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:40.206399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 679/1324] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:40.211610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a844b33-3a5f-4db3-b1d0-f1a838a7a7ef None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:40.211997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a844b33-3a5f-4db3-b1d0-f1a838a7a7ef None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:40.212177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a844b33-3a5f-4db3-b1d0-f1a838a7a7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:40.277247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5a844b33-3a5f-4db3-b1d0-f1a838a7a7ef None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:40.279069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5a844b33-3a5f-4db3-b1d0-f1a838a7a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:40.374583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 646/1325] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:40.410925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644e117-6a9c-4a02-b08f-6b7d604dedc9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:40.412495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9644e117-6a9c-4a02-b08f-6b7d604dedc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:40.514991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644e117-6a9c-4a02-b08f-6b7d604dedc9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:40.586475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:40.587142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:40.587427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:40.587703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:40.589317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:40.606336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:40.698077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:40.700230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d7677-bb3a-4e5d-bb70-c464f1ff0036 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:40.718813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 680/1326] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:43.048592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486386a4-5661-470a-b4e7-0276a5768fb2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:43.049165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486386a4-5661-470a-b4e7-0276a5768fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:43.049165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486386a4-5661-470a-b4e7-0276a5768fb2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:43.050695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 647/1327] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:43.055870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cc7a7-f10b-46b1-be52-e76debb23a5e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:43.056259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cc7a7-f10b-46b1-be52-e76debb23a5e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:43.056438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cc7a7-f10b-46b1-be52-e76debb23a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:43.123071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3f9cc7a7-f10b-46b1-be52-e76debb23a5e None None] MFA Rules not processed for user `350c85611b984015a23d8c9c7772914f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:43.124299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3f9cc7a7-f10b-46b1-be52-e76debb23a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:43.189274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:43.226532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 681/1328] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:43.265369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486386a4-5661-470a-b4e7-0276a5768fb2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:43.267086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-486386a4-5661-470a-b4e7-0276a5768fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:43.302984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:43.376424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486386a4-5661-470a-b4e7-0276a5768fb2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:43.449669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:43.450434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:43.450727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:43.451038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:43.452667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:43.531086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:43.563502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:43.564951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6f11-f767-4765-b509-b79e1ec60280 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:43.582746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 648/1329] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:46.103073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3610c-7891-43a1-ae31-ffdc2d4cc57c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:46.103535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3610c-7891-43a1-ae31-ffdc2d4cc57c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:46.103740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3610c-7891-43a1-ae31-ffdc2d4cc57c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:46.105002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 682/1330] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:46.110919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49ab8d-f56c-41cc-9969-21dd96ccf84e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:46.111397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49ab8d-f56c-41cc-9969-21dd96ccf84e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:46.111521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49ab8d-f56c-41cc-9969-21dd96ccf84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:46.178330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dd49ab8d-f56c-41cc-9969-21dd96ccf84e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:46.180198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dd49ab8d-f56c-41cc-9969-21dd96ccf84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:46.277489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 649/1331] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:46.314343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3610c-7891-43a1-ae31-ffdc2d4cc57c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:46.316832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-01f3610c-7891-43a1-ae31-ffdc2d4cc57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:46.550754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3610c-7891-43a1-ae31-ffdc2d4cc57c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:46.624546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:46.625197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:46.625480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:46.625755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:46.627566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:46.654334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:46.738815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:46.740316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23329a-aba0-4121-8a38-cb59bd26c2aa service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:46.759893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 683/1332] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:49.100410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b1b2-1f29-441f-8f8f-858feb87bb57 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:49.100962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b1b2-1f29-441f-8f8f-858feb87bb57 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:49.100962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b1b2-1f29-441f-8f8f-858feb87bb57 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:49.102337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 650/1333] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:49.108224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2b913-9655-4711-9a05-23ebb375fa56 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:49.108649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2b913-9655-4711-9a05-23ebb375fa56 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:49.108765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2b913-9655-4711-9a05-23ebb375fa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:49.173567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7cd2b913-9655-4711-9a05-23ebb375fa56 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:49.174767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7cd2b913-9655-4711-9a05-23ebb375fa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:49.248841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:49.271561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 684/1334] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:49.311639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b1b2-1f29-441f-8f8f-858feb87bb57 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:49.313212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d1e9b1b2-1f29-441f-8f8f-858feb87bb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:49.352172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:49.415215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b1b2-1f29-441f-8f8f-858feb87bb57 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:49.487367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:49.488004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:49.488286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:49.488563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:49.490166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:49.571849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:49.596652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:49.598086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980f343-1d80-4767-84a2-29093ab0fd48 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:49.616242 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 651/1335] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:52.329048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49f891-22e7-4875-8383-41c34faea721 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:52.329574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49f891-22e7-4875-8383-41c34faea721 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:52.329688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49f891-22e7-4875-8383-41c34faea721 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:52.331019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 685/1336] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:52.336183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5948c-7544-45f5-bbb2-22a4c56c6bac None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:52.336626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5948c-7544-45f5-bbb2-22a4c56c6bac None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:52.336757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5948c-7544-45f5-bbb2-22a4c56c6bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:52.401290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-75e5948c-7544-45f5-bbb2-22a4c56c6bac None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:52.402500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-75e5948c-7544-45f5-bbb2-22a4c56c6bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:52.497543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 652/1337] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:52.534037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49f891-22e7-4875-8383-41c34faea721 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:52.535590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bb49f891-22e7-4875-8383-41c34faea721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:52.637354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49f891-22e7-4875-8383-41c34faea721 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:52.710865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:52.711532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:52.711815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:52.712091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:52.713751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:52.749085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:52.826751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:52.828228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78269eb-9c21-44e1-be03-e2fb2d8a0ff4 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:52.846033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 686/1338] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:55:57.073726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3655a9e-e5ab-48b6-9b49-2886c51b0f71 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:57.074276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3655a9e-e5ab-48b6-9b49-2886c51b0f71 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:57.074276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3655a9e-e5ab-48b6-9b49-2886c51b0f71 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:57.076163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 653/1339] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:55:57.081464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5034d1-9b30-4558-9dcd-bd3898fc9c09 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:57.081890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5034d1-9b30-4558-9dcd-bd3898fc9c09 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:57.082028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5034d1-9b30-4558-9dcd-bd3898fc9c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:57.154439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4f5034d1-9b30-4558-9dcd-bd3898fc9c09 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:57.155600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4f5034d1-9b30-4558-9dcd-bd3898fc9c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:57.249572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:55:57.263538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 687/1340] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:55:57.317860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2897-b753-4ad6-9720-ca9d4a9b4a69 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:57.318247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2897-b753-4ad6-9720-ca9d4a9b4a69 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:57.318537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2897-b753-4ad6-9720-ca9d4a9b4a69 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:57.319976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 654/1341] 10.222.0.41 () {54 vars in 964 bytes} [Mon Jun 6 11:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:55:57.326483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5024c-658f-4e1e-873c-081f84ffc89b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:57.326773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5024c-658f-4e1e-873c-081f84ffc89b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:57.327112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5024c-658f-4e1e-873c-081f84ffc89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:57.395241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f7a5024c-658f-4e1e-873c-081f84ffc89b None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:55:57.396443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f7a5024c-658f-4e1e-873c-081f84ffc89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:55:57.473293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 688/1342] 10.222.0.41 () {58 vars in 1061 bytes} [Mon Jun 6 11:55:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 11:55:57.482706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2897-b753-4ad6-9720-ca9d4a9b4a69 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:55:57.494443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2897-b753-4ad6-9720-ca9d4a9b4a69 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:55:57.498552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8c1cd-3b37-4a36-aeae-212801cec9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:55:57.499228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8c1cd-3b37-4a36-aeae-212801cec9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:55:57.499520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8c1cd-3b37-4a36-aeae-212801cec9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:55:57.499802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8c1cd-3b37-4a36-aeae-212801cec9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:55:57.509075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8c1cd-3b37-4a36-aeae-212801cec9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:55:57.510520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8c1cd-3b37-4a36-aeae-212801cec9d4 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:55:57.521850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 655/1343] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:00.985922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da929-12db-44c3-b916-0a37eca651fd None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:00.986498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da929-12db-44c3-b916-0a37eca651fd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:00.986498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da929-12db-44c3-b916-0a37eca651fd None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:00.988145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 689/1344] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:00.993361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e343c4b-a9b9-447c-b17b-457b14ef5e49 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:00.993793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e343c4b-a9b9-447c-b17b-457b14ef5e49 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:00.993975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e343c4b-a9b9-447c-b17b-457b14ef5e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:01.023264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5e343c4b-a9b9-447c-b17b-457b14ef5e49 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:01.024464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5e343c4b-a9b9-447c-b17b-457b14ef5e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:01.042746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:01.102364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 656/1345] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 11:56:01.139494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4da929-12db-44c3-b916-0a37eca651fd None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:01.150637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4da929-12db-44c3-b916-0a37eca651fd None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:01.154968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a114355-47a1-4ab2-8ca3-7d9c5fa96405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:01.155612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114355-47a1-4ab2-8ca3-7d9c5fa96405 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:01.155894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114355-47a1-4ab2-8ca3-7d9c5fa96405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:01.156173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114355-47a1-4ab2-8ca3-7d9c5fa96405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:01.165618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114355-47a1-4ab2-8ca3-7d9c5fa96405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:01.167097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114355-47a1-4ab2-8ca3-7d9c5fa96405 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:01.177224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 690/1346] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:05.630460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e84660-7f55-4846-a77a-953c29454ddc None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:05.631054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e84660-7f55-4846-a77a-953c29454ddc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:05.631161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e84660-7f55-4846-a77a-953c29454ddc None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:05.632549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 657/1347] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:05.637785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ed0dd-f8d3-4841-8716-5f6535e5b5b1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:05.638230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ed0dd-f8d3-4841-8716-5f6535e5b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:05.638354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ed0dd-f8d3-4841-8716-5f6535e5b5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:05.657080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-877ed0dd-f8d3-4841-8716-5f6535e5b5b1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:05.658318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-877ed0dd-f8d3-4841-8716-5f6535e5b5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:05.666190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 691/1348] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:05.704772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e84660-7f55-4846-a77a-953c29454ddc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:05.715821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e84660-7f55-4846-a77a-953c29454ddc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:05.719602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf82aab-6675-4dc3-a48c-296e195c4153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:05.720256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf82aab-6675-4dc3-a48c-296e195c4153 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:05.720545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf82aab-6675-4dc3-a48c-296e195c4153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:05.720831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf82aab-6675-4dc3-a48c-296e195c4153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:05.730431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf82aab-6675-4dc3-a48c-296e195c4153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:05.731955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf82aab-6675-4dc3-a48c-296e195c4153 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:05.742164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 658/1349] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:11.228482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c5806-7283-442d-82c8-f2881a61cbd1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:11.228994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c5806-7283-442d-82c8-f2881a61cbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:11.229106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c5806-7283-442d-82c8-f2881a61cbd1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:11.230734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 692/1350] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 11:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:11.240524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395bbe-6ca3-492e-b230-c75767eaf667 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:11.240968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395bbe-6ca3-492e-b230-c75767eaf667 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:11.241087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395bbe-6ca3-492e-b230-c75767eaf667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:11.311304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fd395bbe-6ca3-492e-b230-c75767eaf667 None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:11.312508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd395bbe-6ca3-492e-b230-c75767eaf667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:11.400849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 659/1351] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 11:56:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:15.652265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=66 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:15.786760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179831-84cc-4bd0-9275-d585bf769ba4 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:15.787320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179831-84cc-4bd0-9275-d585bf769ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:15.787605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179831-84cc-4bd0-9275-d585bf769ba4 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:15.788967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 693/1352] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 11:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:15.794037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592280a-3225-4d61-bc93-b1e21e278fbf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:15.794927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592280a-3225-4d61-bc93-b1e21e278fbf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:15.795458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592280a-3225-4d61-bc93-b1e21e278fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:15.816499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f592280a-3225-4d61-bc93-b1e21e278fbf None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:15.817970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f592280a-3225-4d61-bc93-b1e21e278fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:15.827714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 660/1353] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 11:56:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:17.310525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997d8be-7118-4204-90b0-cd934716b83a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:17.311105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997d8be-7118-4204-90b0-cd934716b83a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:17.311105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997d8be-7118-4204-90b0-cd934716b83a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:17.312873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 694/1354] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 11:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:17.318620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114ddc6-be9d-46d5-9fbb-5229ede03dd5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:17.319172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114ddc6-be9d-46d5-9fbb-5229ede03dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:17.319368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114ddc6-be9d-46d5-9fbb-5229ede03dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:17.343928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e114ddc6-be9d-46d5-9fbb-5229ede03dd5 None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:17.346002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e114ddc6-be9d-46d5-9fbb-5229ede03dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:17.355559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 661/1355] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 11:56:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:21.250283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ef84d-0919-4d01-bf96-ce68265083e8 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:21.250808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ef84d-0919-4d01-bf96-ce68265083e8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:21.250808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ef84d-0919-4d01-bf96-ce68265083e8 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:21.252134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 695/1356] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:56:21.257717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b67f2-aa21-4ccd-9b01-a32f456cd78d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:21.258157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b67f2-aa21-4ccd-9b01-a32f456cd78d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:21.258314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b67f2-aa21-4ccd-9b01-a32f456cd78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:21.277563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dc1b67f2-aa21-4ccd-9b01-a32f456cd78d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:21.279036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dc1b67f2-aa21-4ccd-9b01-a32f456cd78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:21.287217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 662/1357] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:21.342519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcd530-ff85-4dc5-8143-fff4108350b4 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:21.343119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcd530-ff85-4dc5-8143-fff4108350b4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:21.343624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcd530-ff85-4dc5-8143-fff4108350b4 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:21.345452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 696/1358] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 11:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:21.356639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4349-137b-4572-9e59-298cf5dc38cc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:21.357086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4349-137b-4572-9e59-298cf5dc38cc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:21.357476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4349-137b-4572-9e59-298cf5dc38cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:21.438259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-879e4349-137b-4572-9e59-298cf5dc38cc None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:21.439589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-879e4349-137b-4572-9e59-298cf5dc38cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:21.445885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:21.527439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 663/1359] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 11:56:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:21.535411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcd530-ff85-4dc5-8143-fff4108350b4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:21.550479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcd530-ff85-4dc5-8143-fff4108350b4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:21.555993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcd16c-f951-4fa7-aba2-79a1743e0507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:21.556703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcd16c-f951-4fa7-aba2-79a1743e0507 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:21.557183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcd16c-f951-4fa7-aba2-79a1743e0507 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:21.557627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcd16c-f951-4fa7-aba2-79a1743e0507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:21.567017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcd16c-f951-4fa7-aba2-79a1743e0507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:21.568567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcd16c-f951-4fa7-aba2-79a1743e0507 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:21.578557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 697/1360] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:24.083583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58656-79e5-44cc-a882-75946aa336c6 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:24.084125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58656-79e5-44cc-a882-75946aa336c6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:24.084209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58656-79e5-44cc-a882-75946aa336c6 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:24.085747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 664/1361] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:24.090879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64248ae-1eab-409e-b182-3f4d0d330efb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:24.091693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64248ae-1eab-409e-b182-3f4d0d330efb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:24.092131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64248ae-1eab-409e-b182-3f4d0d330efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:24.110661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a64248ae-1eab-409e-b182-3f4d0d330efb None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:24.111924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a64248ae-1eab-409e-b182-3f4d0d330efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:24.120315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 698/1362] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:24.167882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58656-79e5-44cc-a882-75946aa336c6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:24.188342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58656-79e5-44cc-a882-75946aa336c6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:24.193718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc51cf7-f9ff-456d-871b-a99d13809c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:24.194372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc51cf7-f9ff-456d-871b-a99d13809c68 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:24.194692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc51cf7-f9ff-456d-871b-a99d13809c68 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:24.195034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc51cf7-f9ff-456d-871b-a99d13809c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:24.203908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc51cf7-f9ff-456d-871b-a99d13809c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:24.205525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc51cf7-f9ff-456d-871b-a99d13809c68 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:24.215538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 665/1363] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 11:56:26.816822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac8f5-a6e1-49c5-9c8d-beaf3aafdbbb None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:26.817321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac8f5-a6e1-49c5-9c8d-beaf3aafdbbb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:26.817462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac8f5-a6e1-49c5-9c8d-beaf3aafdbbb None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:26.819156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 699/1364] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:26.824893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900812e-c2f6-48c8-9085-cbc65a497da1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:26.825317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900812e-c2f6-48c8-9085-cbc65a497da1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:26.826238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900812e-c2f6-48c8-9085-cbc65a497da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:26.845416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6900812e-c2f6-48c8-9085-cbc65a497da1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:26.846614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6900812e-c2f6-48c8-9085-cbc65a497da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:26.854550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 666/1365] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:26.904105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf229df-7fe2-40f9-8c8c-fc707db6f15a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:26.907574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf229df-7fe2-40f9-8c8c-fc707db6f15a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:26.908265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf229df-7fe2-40f9-8c8c-fc707db6f15a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:26.911494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 700/1366] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 11:56:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:26.919086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556646a-80af-4368-8073-9c5ee039860c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:26.919613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556646a-80af-4368-8073-9c5ee039860c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:26.920063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556646a-80af-4368-8073-9c5ee039860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:27.080982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f556646a-80af-4368-8073-9c5ee039860c None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:27.082119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f556646a-80af-4368-8073-9c5ee039860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:27.091391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 667/1367] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 11:56:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:27.099715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf229df-7fe2-40f9-8c8c-fc707db6f15a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:27.114046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf229df-7fe2-40f9-8c8c-fc707db6f15a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:27.119746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5d1f-2e71-4270-af30-641fe8c34fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:27.120383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5d1f-2e71-4270-af30-641fe8c34fba service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:27.120674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5d1f-2e71-4270-af30-641fe8c34fba service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:27.121028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5d1f-2e71-4270-af30-641fe8c34fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:27.129111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5d1f-2e71-4270-af30-641fe8c34fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:27.130712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5d1f-2e71-4270-af30-641fe8c34fba service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:27.140192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 701/1368] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:29.508578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450594-8108-4200-b85c-1ea7929c5097 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:29.511015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450594-8108-4200-b85c-1ea7929c5097 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:29.511525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f450594-8108-4200-b85c-1ea7929c5097 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:29.513130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 668/1369] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:29.519434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba7e3-4c27-4831-822f-44522822d934 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:29.519899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba7e3-4c27-4831-822f-44522822d934 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:29.519982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba7e3-4c27-4831-822f-44522822d934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:29.537772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5bfba7e3-4c27-4831-822f-44522822d934 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:29.539009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5bfba7e3-4c27-4831-822f-44522822d934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:29.547100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 702/1370] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:29.592508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450594-8108-4200-b85c-1ea7929c5097 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:29.608161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f450594-8108-4200-b85c-1ea7929c5097 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:29.614248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd512363-382f-4a5f-8cec-8dd650fab7e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:29.614958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512363-382f-4a5f-8cec-8dd650fab7e9 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:29.615245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512363-382f-4a5f-8cec-8dd650fab7e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:29.615779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512363-382f-4a5f-8cec-8dd650fab7e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:29.625835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd512363-382f-4a5f-8cec-8dd650fab7e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:29.627518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd512363-382f-4a5f-8cec-8dd650fab7e9 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:29.638380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 669/1371] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:29.715330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7338-fe12-4b56-9843-d89d1781caac None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:29.715782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7338-fe12-4b56-9843-d89d1781caac None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:29.716165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7338-fe12-4b56-9843-d89d1781caac None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:29.717717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 703/1372] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:29.724344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a1929-19ce-422b-8dc2-1f377b0b3021 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:29.724633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a1929-19ce-422b-8dc2-1f377b0b3021 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:29.725122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a1929-19ce-422b-8dc2-1f377b0b3021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:29.745002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-860a1929-19ce-422b-8dc2-1f377b0b3021 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:29.746272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-860a1929-19ce-422b-8dc2-1f377b0b3021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:29.754910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 670/1373] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:30.071044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0c44-e7a6-41ed-9d97-da1fc412b02c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:30.071430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0c44-e7a6-41ed-9d97-da1fc412b02c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:30.071761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0c44-e7a6-41ed-9d97-da1fc412b02c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:30.073631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 704/1374] 10.222.0.41 () {54 vars in 963 bytes} [Mon Jun 6 11:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:30.079464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bef8-6f61-4e66-a1bd-3924db0f162c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:30.079758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bef8-6f61-4e66-a1bd-3924db0f162c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:30.080066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bef8-6f61-4e66-a1bd-3924db0f162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:30.131407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9a68bef8-6f61-4e66-a1bd-3924db0f162c None None] MFA Rules not processed for user `106467a9f0644177872555b7998599d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:30.133183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9a68bef8-6f61-4e66-a1bd-3924db0f162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:30.201103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 671/1375] 10.222.0.41 () {58 vars in 1060 bytes} [Mon Jun 6 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:30.208728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0c44-e7a6-41ed-9d97-da1fc412b02c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:30.220114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0c44-e7a6-41ed-9d97-da1fc412b02c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:30.223807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a409dd-8a84-4fde-ac1f-81b0a93f4398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:30.224450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a409dd-8a84-4fde-ac1f-81b0a93f4398 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:30.224734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a409dd-8a84-4fde-ac1f-81b0a93f4398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:30.225009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a409dd-8a84-4fde-ac1f-81b0a93f4398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:30.235645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a409dd-8a84-4fde-ac1f-81b0a93f4398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:30.237347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a409dd-8a84-4fde-ac1f-81b0a93f4398 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:30.247689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 705/1376] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:31.963421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1799a-39f8-4015-add0-9361b6a132d3 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:31.963973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1799a-39f8-4015-add0-9361b6a132d3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:31.963973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1799a-39f8-4015-add0-9361b6a132d3 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:31.965408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 672/1377] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:31.972053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d29b5e-6994-4c0d-8bcc-7cb3bca0a144 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:31.972642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d29b5e-6994-4c0d-8bcc-7cb3bca0a144 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:31.972642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d29b5e-6994-4c0d-8bcc-7cb3bca0a144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:31.991875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-52d29b5e-6994-4c0d-8bcc-7cb3bca0a144 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:31.993128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-52d29b5e-6994-4c0d-8bcc-7cb3bca0a144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:32.002213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 706/1378] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:32.048694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1799a-39f8-4015-add0-9361b6a132d3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:32.063405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1799a-39f8-4015-add0-9361b6a132d3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:32.068995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43889-d3e4-40fa-8719-c465334bc77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:32.069659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43889-d3e4-40fa-8719-c465334bc77f service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:32.069940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43889-d3e4-40fa-8719-c465334bc77f service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:32.070214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43889-d3e4-40fa-8719-c465334bc77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:32.079855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d43889-d3e4-40fa-8719-c465334bc77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:32.081547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d43889-d3e4-40fa-8719-c465334bc77f service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:32.091931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 673/1379] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:34.471976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5636-19b3-44e3-9a03-91b2c24dd8ca None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:34.472548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5636-19b3-44e3-9a03-91b2c24dd8ca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:34.472548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5636-19b3-44e3-9a03-91b2c24dd8ca None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:34.473891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 707/1380] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:34.479496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e8d79-59b0-4d53-92cf-fa0cb2b843aa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:34.479928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e8d79-59b0-4d53-92cf-fa0cb2b843aa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:34.480147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e8d79-59b0-4d53-92cf-fa0cb2b843aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:34.488455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:34.500330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-703e8d79-59b0-4d53-92cf-fa0cb2b843aa None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:34.502552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-703e8d79-59b0-4d53-92cf-fa0cb2b843aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:34.510887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 674/1381] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:34.556458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5636-19b3-44e3-9a03-91b2c24dd8ca None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:34.571872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5636-19b3-44e3-9a03-91b2c24dd8ca None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:34.577346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c211d2a-4b1d-408a-8d1b-04b24d4474b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:34.577985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c211d2a-4b1d-408a-8d1b-04b24d4474b3 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:34.578271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c211d2a-4b1d-408a-8d1b-04b24d4474b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:34.578600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c211d2a-4b1d-408a-8d1b-04b24d4474b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:34.587663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c211d2a-4b1d-408a-8d1b-04b24d4474b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:34.589254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c211d2a-4b1d-408a-8d1b-04b24d4474b3 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:34.599719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 708/1382] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:36.926699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fba87f-5607-499d-90b3-4e2b91d5e92f None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:36.927250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fba87f-5607-499d-90b3-4e2b91d5e92f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:36.927478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fba87f-5607-499d-90b3-4e2b91d5e92f None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:36.929012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 675/1383] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:36.934371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17c5d-3225-42ce-800f-eb442558d28a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:36.934825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17c5d-3225-42ce-800f-eb442558d28a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:36.935044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17c5d-3225-42ce-800f-eb442558d28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:36.962049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bda17c5d-3225-42ce-800f-eb442558d28a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:36.963398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bda17c5d-3225-42ce-800f-eb442558d28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:36.974918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 709/1384] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:37.017675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fba87f-5607-499d-90b3-4e2b91d5e92f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:37.033609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fba87f-5607-499d-90b3-4e2b91d5e92f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:37.039077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d528701-b50d-4d12-8e5d-a07fbeb0692a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:37.039726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d528701-b50d-4d12-8e5d-a07fbeb0692a service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:37.040006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d528701-b50d-4d12-8e5d-a07fbeb0692a service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:37.040453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d528701-b50d-4d12-8e5d-a07fbeb0692a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:37.050575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d528701-b50d-4d12-8e5d-a07fbeb0692a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:37.052201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d528701-b50d-4d12-8e5d-a07fbeb0692a service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:37.061979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 676/1385] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:39.340554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257be60-3186-4de4-b3b5-9e8be945c85c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:39.341175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257be60-3186-4de4-b3b5-9e8be945c85c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:39.341175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257be60-3186-4de4-b3b5-9e8be945c85c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:39.342476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 710/1386] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:39.348956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d1da6-e274-455a-85bf-ba2224932196 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:39.349375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d1da6-e274-455a-85bf-ba2224932196 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:39.349505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d1da6-e274-455a-85bf-ba2224932196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:39.367485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-521d1da6-e274-455a-85bf-ba2224932196 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:39.368673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-521d1da6-e274-455a-85bf-ba2224932196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:39.377144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 677/1387] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:39.423140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257be60-3186-4de4-b3b5-9e8be945c85c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:39.439410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257be60-3186-4de4-b3b5-9e8be945c85c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:39.445221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739ea66-7c34-49de-bd20-9cf9b6bfc2b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:39.445888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ea66-7c34-49de-bd20-9cf9b6bfc2b4 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:39.446163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ea66-7c34-49de-bd20-9cf9b6bfc2b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:39.446444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ea66-7c34-49de-bd20-9cf9b6bfc2b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:39.455337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ea66-7c34-49de-bd20-9cf9b6bfc2b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:39.457018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ea66-7c34-49de-bd20-9cf9b6bfc2b4 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:39.466831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 711/1388] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:41.801273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78389ac-c53d-448f-9f31-e76b0b97382e None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:41.801970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78389ac-c53d-448f-9f31-e76b0b97382e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:41.801970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78389ac-c53d-448f-9f31-e76b0b97382e None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:41.803571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 678/1389] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:41.809120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7e32d-63d0-4405-b3b7-878c5f533391 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:41.809629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7e32d-63d0-4405-b3b7-878c5f533391 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:41.809629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7e32d-63d0-4405-b3b7-878c5f533391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:41.827870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2ca7e32d-63d0-4405-b3b7-878c5f533391 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:41.829032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2ca7e32d-63d0-4405-b3b7-878c5f533391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:41.836829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 712/1390] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:41.883089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78389ac-c53d-448f-9f31-e76b0b97382e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:41.897985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78389ac-c53d-448f-9f31-e76b0b97382e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:41.903401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ccac5-962d-429e-991d-f6e832fe96bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:41.904043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ccac5-962d-429e-991d-f6e832fe96bb service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:41.904330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ccac5-962d-429e-991d-f6e832fe96bb service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:41.904616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ccac5-962d-429e-991d-f6e832fe96bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:41.913285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ccac5-962d-429e-991d-f6e832fe96bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:41.914907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ccac5-962d-429e-991d-f6e832fe96bb service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:41.924882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 679/1391] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:44.209254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9112c-9b17-4c67-8037-7417a9ad3ad5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:44.209881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9112c-9b17-4c67-8037-7417a9ad3ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:44.209881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9112c-9b17-4c67-8037-7417a9ad3ad5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:44.211235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 713/1392] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:44.216518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d0b38-de8f-44d5-b6cd-0181df9a3ac2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:44.216956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d0b38-de8f-44d5-b6cd-0181df9a3ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:44.217073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d0b38-de8f-44d5-b6cd-0181df9a3ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:44.235352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-065d0b38-de8f-44d5-b6cd-0181df9a3ac2 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:44.236720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-065d0b38-de8f-44d5-b6cd-0181df9a3ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:44.244796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 680/1393] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:44.290449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9112c-9b17-4c67-8037-7417a9ad3ad5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:44.305287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9112c-9b17-4c67-8037-7417a9ad3ad5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:44.310941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92107a76-878f-42bf-b63d-e3735b186f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:44.311570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92107a76-878f-42bf-b63d-e3735b186f6a service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:44.311857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92107a76-878f-42bf-b63d-e3735b186f6a service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:44.312131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92107a76-878f-42bf-b63d-e3735b186f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:44.321000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92107a76-878f-42bf-b63d-e3735b186f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:44.322591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92107a76-878f-42bf-b63d-e3735b186f6a service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:44.333019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 714/1394] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:46.144868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d8404-651b-49e4-8503-4fa643dfc5f6 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.145422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d8404-651b-49e4-8503-4fa643dfc5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.145624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d8404-651b-49e4-8503-4fa643dfc5f6 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.147069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 681/1395] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:46.152495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4bc89-9e77-4b8b-9089-42848a249a5f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.152914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4bc89-9e77-4b8b-9089-42848a249a5f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.153086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4bc89-9e77-4b8b-9089-42848a249a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.159933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=52 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:46.172683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a2e4bc89-9e77-4b8b-9089-42848a249a5f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:46.174671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a2e4bc89-9e77-4b8b-9089-42848a249a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:46.182676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 715/1396] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:46.239847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d72131-a40c-4364-be60-7312cb1136e6 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.240141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d72131-a40c-4364-be60-7312cb1136e6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.240427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d72131-a40c-4364-be60-7312cb1136e6 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.241960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 682/1397] 10.222.0.41 () {54 vars in 962 bytes} [Mon Jun 6 11:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:46.246927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345892d-79d1-46fd-a341-7761a2ed1a4c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.247227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345892d-79d1-46fd-a341-7761a2ed1a4c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.247511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345892d-79d1-46fd-a341-7761a2ed1a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.301069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a345892d-79d1-46fd-a341-7761a2ed1a4c None None] MFA Rules not processed for user `d0fa7ced305e4d2fa0d791675f839453`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:46.302325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a345892d-79d1-46fd-a341-7761a2ed1a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:46.364265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 716/1398] 10.222.0.41 () {58 vars in 1059 bytes} [Mon Jun 6 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:46.372271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d72131-a40c-4364-be60-7312cb1136e6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:46.383638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d72131-a40c-4364-be60-7312cb1136e6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:46.387182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b104f-ced1-410d-8acf-b0977521c27d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:46.387877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b104f-ced1-410d-8acf-b0977521c27d service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.388231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b104f-ced1-410d-8acf-b0977521c27d service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.388590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b104f-ced1-410d-8acf-b0977521c27d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.397689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b104f-ced1-410d-8acf-b0977521c27d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:46.399215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b104f-ced1-410d-8acf-b0977521c27d service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:46.408887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 683/1399] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:46.615726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75015554-a8a2-4db3-8ad3-1c57e823074c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.615921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75015554-a8a2-4db3-8ad3-1c57e823074c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.616197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75015554-a8a2-4db3-8ad3-1c57e823074c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.617850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 717/1400] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:46.623203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e61a-351a-4b6f-96e4-5de4ad7464d9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.623505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e61a-351a-4b6f-96e4-5de4ad7464d9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.623784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e61a-351a-4b6f-96e4-5de4ad7464d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.642396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7424e61a-351a-4b6f-96e4-5de4ad7464d9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:46.643626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7424e61a-351a-4b6f-96e4-5de4ad7464d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:46.652666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 684/1401] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:46.699070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75015554-a8a2-4db3-8ad3-1c57e823074c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:46.713606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75015554-a8a2-4db3-8ad3-1c57e823074c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:46.719132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798548db-5bef-46d4-8a30-f6a80ffd4523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:46.719770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798548db-5bef-46d4-8a30-f6a80ffd4523 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.720586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798548db-5bef-46d4-8a30-f6a80ffd4523 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.720867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798548db-5bef-46d4-8a30-f6a80ffd4523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:46.733058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798548db-5bef-46d4-8a30-f6a80ffd4523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:46.734671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798548db-5bef-46d4-8a30-f6a80ffd4523 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:46.745828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 718/1402] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:46.965023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986936c-b4f1-421b-a6af-fc05fbe42d9b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:46.965699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986936c-b4f1-421b-a6af-fc05fbe42d9b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:46.966088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986936c-b4f1-421b-a6af-fc05fbe42d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:47.024594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f986936c-b4f1-421b-a6af-fc05fbe42d9b None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:47.025891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f986936c-b4f1-421b-a6af-fc05fbe42d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:47.096453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 685/1403] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:47.140805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7380f-0a89-41b7-a0b7-8cf7d872af9c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:47.141176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7380f-0a89-41b7-a0b7-8cf7d872af9c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:47.141530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7380f-0a89-41b7-a0b7-8cf7d872af9c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:47.143592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 719/1404] 10.222.0.41 () {54 vars in 942 bytes} [Mon Jun 6 11:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:47.150053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285800b6-c3fa-4eae-906c-51c2d0b3569a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:47.150528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285800b6-c3fa-4eae-906c-51c2d0b3569a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:47.150634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285800b6-c3fa-4eae-906c-51c2d0b3569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:47.200340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-285800b6-c3fa-4eae-906c-51c2d0b3569a None None] MFA Rules not processed for user `0a914be52bf745d4b4647f482460e734`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:47.201556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-285800b6-c3fa-4eae-906c-51c2d0b3569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:47.246161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:47.278760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 686/1405] 10.222.0.41 () {58 vars in 1039 bytes} [Mon Jun 6 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:47.289488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7380f-0a89-41b7-a0b7-8cf7d872af9c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:47.303469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7380f-0a89-41b7-a0b7-8cf7d872af9c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:47.308581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcc36b-4a4a-49d4-857c-659b85424b08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:47.309258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc36b-4a4a-49d4-857c-659b85424b08 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:47.309546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc36b-4a4a-49d4-857c-659b85424b08 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:47.309852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc36b-4a4a-49d4-857c-659b85424b08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:47.318324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcc36b-4a4a-49d4-857c-659b85424b08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:47.319983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcc36b-4a4a-49d4-857c-659b85424b08 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:47.328472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 720/1406] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 11:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 11:56:49.079275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06478365-05cb-4cd0-950b-ed73ebd58bf9 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:49.080489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06478365-05cb-4cd0-950b-ed73ebd58bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:49.080489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06478365-05cb-4cd0-950b-ed73ebd58bf9 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:49.081904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 687/1407] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:56:49.087094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb102453-5dc7-4ff9-aa0a-f0cbac74affd None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:49.087545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb102453-5dc7-4ff9-aa0a-f0cbac74affd None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:49.087670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb102453-5dc7-4ff9-aa0a-f0cbac74affd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:49.109230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cb102453-5dc7-4ff9-aa0a-f0cbac74affd None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:56:49.110455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cb102453-5dc7-4ff9-aa0a-f0cbac74affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:56:49.119042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 721/1408] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:56:49.165929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06478365-05cb-4cd0-950b-ed73ebd58bf9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:56:49.180323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06478365-05cb-4cd0-950b-ed73ebd58bf9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:56:49.185636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa3d4c-4b2e-4038-9fe4-564ebf5da0fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:56:49.186267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa3d4c-4b2e-4038-9fe4-564ebf5da0fd service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:56:49.186548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa3d4c-4b2e-4038-9fe4-564ebf5da0fd service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:56:49.186823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa3d4c-4b2e-4038-9fe4-564ebf5da0fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:56:49.200971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa3d4c-4b2e-4038-9fe4-564ebf5da0fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:56:49.202602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa3d4c-4b2e-4038-9fe4-564ebf5da0fd service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:56:49.213854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 688/1409] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:56:59.100316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=55 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:56:59.209575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=18 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:57:10.936655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676ee1-2962-462c-a136-ee663fc29bab None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:10.937296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676ee1-2962-462c-a136-ee663fc29bab None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:10.937296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676ee1-2962-462c-a136-ee663fc29bab None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:10.938608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 722/1410] 10.222.0.41 () {54 vars in 918 bytes} [Mon Jun 6 11:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:10.944527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd678f-e3b9-4ce4-9ed0-5aac1171bcf2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:10.945186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd678f-e3b9-4ce4-9ed0-5aac1171bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:10.945372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd678f-e3b9-4ce4-9ed0-5aac1171bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:10.964291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-44fd678f-e3b9-4ce4-9ed0-5aac1171bcf2 None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:10.965502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-44fd678f-e3b9-4ce4-9ed0-5aac1171bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:10.974459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 689/1411] 10.222.0.41 () {58 vars in 1015 bytes} [Mon Jun 6 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:10.989253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876f989-da10-4521-b254-9961eddc4589 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:10.989547 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876f989-da10-4521-b254-9961eddc4589 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:10.989831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876f989-da10-4521-b254-9961eddc4589 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:10.991256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 723/1412] 10.222.0.41 () {54 vars in 918 bytes} [Mon Jun 6 11:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:10.995898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe02aa-618c-4f71-8f6b-617b9acc27c8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:10.996192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe02aa-618c-4f71-8f6b-617b9acc27c8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:10.996475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe02aa-618c-4f71-8f6b-617b9acc27c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:11.014160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fbfe02aa-618c-4f71-8f6b-617b9acc27c8 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:11.015396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fbfe02aa-618c-4f71-8f6b-617b9acc27c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:11.025466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 690/1413] 10.222.0.41 () {58 vars in 1015 bytes} [Mon Jun 6 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:13.741752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7cb21-8e7e-41f1-bfff-9b8371e3045c None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:13.742274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7cb21-8e7e-41f1-bfff-9b8371e3045c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:13.742394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7cb21-8e7e-41f1-bfff-9b8371e3045c None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:13.743731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 724/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:13.749309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a25e9f-bfba-43fa-b92d-0a6defd636ed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:13.749709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a25e9f-bfba-43fa-b92d-0a6defd636ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:13.749866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a25e9f-bfba-43fa-b92d-0a6defd636ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:13.768832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c7a25e9f-bfba-43fa-b92d-0a6defd636ed None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:13.770602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c7a25e9f-bfba-43fa-b92d-0a6defd636ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:13.778425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 691/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:13.821751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7cb21-8e7e-41f1-bfff-9b8371e3045c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:13.837408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7cb21-8e7e-41f1-bfff-9b8371e3045c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:13.842713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ca31-f0e1-4be1-aba4-25883ce85498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:13.843370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ca31-f0e1-4be1-aba4-25883ce85498 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:13.843656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ca31-f0e1-4be1-aba4-25883ce85498 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:13.843936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ca31-f0e1-4be1-aba4-25883ce85498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:13.852250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ca31-f0e1-4be1-aba4-25883ce85498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:13.853826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ca31-f0e1-4be1-aba4-25883ce85498 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:13.863037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 725/1416] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:14.842872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274b4c-8906-4317-b138-44049a6ea1d1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:14.843439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274b4c-8906-4317-b138-44049a6ea1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:14.843439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274b4c-8906-4317-b138-44049a6ea1d1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:14.844820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 692/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:14.850526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f90747-b5a5-4701-ab5d-d2a6f04e2f54 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:14.851003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f90747-b5a5-4701-ab5d-d2a6f04e2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:14.851126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f90747-b5a5-4701-ab5d-d2a6f04e2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:14.869624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-66f90747-b5a5-4701-ab5d-d2a6f04e2f54 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:14.870822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-66f90747-b5a5-4701-ab5d-d2a6f04e2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:14.878774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 726/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:15.063413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1274b4c-8906-4317-b138-44049a6ea1d1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:15.077601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1274b4c-8906-4317-b138-44049a6ea1d1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:15.083571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbec6-d599-4e5a-aecd-fcec4760614e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:15.084204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbec6-d599-4e5a-aecd-fcec4760614e service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:15.084494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbec6-d599-4e5a-aecd-fcec4760614e service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:15.084774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbec6-d599-4e5a-aecd-fcec4760614e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:15.092737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfbec6-d599-4e5a-aecd-fcec4760614e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:15.094368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfbec6-d599-4e5a-aecd-fcec4760614e service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:15.104412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 693/1419] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:16.033646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ead1fa-073c-457b-a80d-cfc242392b3f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:16.034095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ead1fa-073c-457b-a80d-cfc242392b3f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:16.034206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ead1fa-073c-457b-a80d-cfc242392b3f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:16.036234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 727/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:16.041133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c289-f306-4944-84e4-6df6a44822e7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:16.041559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c289-f306-4944-84e4-6df6a44822e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:16.041717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c289-f306-4944-84e4-6df6a44822e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:16.059951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3b30c289-f306-4944-84e4-6df6a44822e7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:16.061143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3b30c289-f306-4944-84e4-6df6a44822e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:16.068863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 694/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:16.223327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ead1fa-073c-457b-a80d-cfc242392b3f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:16.237281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ead1fa-073c-457b-a80d-cfc242392b3f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:16.243638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700add6-5284-4943-b5f6-0dc659ff3add service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:16.244266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700add6-5284-4943-b5f6-0dc659ff3add service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:16.244553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700add6-5284-4943-b5f6-0dc659ff3add service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:16.244833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700add6-5284-4943-b5f6-0dc659ff3add service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:16.253072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700add6-5284-4943-b5f6-0dc659ff3add service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:16.254790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700add6-5284-4943-b5f6-0dc659ff3add service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:16.264043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 728/1422] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:17.167146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33506fe-7e93-472c-86cd-9fd186e10902 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:17.167748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33506fe-7e93-472c-86cd-9fd186e10902 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:17.167748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33506fe-7e93-472c-86cd-9fd186e10902 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:17.169088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 695/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:17.174050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99c9c-9ff5-4f04-bfc2-923baf4d88f1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:17.174467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99c9c-9ff5-4f04-bfc2-923baf4d88f1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:17.174615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99c9c-9ff5-4f04-bfc2-923baf4d88f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:17.193401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-56d99c9c-9ff5-4f04-bfc2-923baf4d88f1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:17.194596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-56d99c9c-9ff5-4f04-bfc2-923baf4d88f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:17.202318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 729/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:17.349959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33506fe-7e93-472c-86cd-9fd186e10902 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:17.365265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33506fe-7e93-472c-86cd-9fd186e10902 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:17.370566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23319e84-dd67-4ebf-8156-318a9ea42694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:17.371492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23319e84-dd67-4ebf-8156-318a9ea42694 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:17.371779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23319e84-dd67-4ebf-8156-318a9ea42694 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:17.372058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23319e84-dd67-4ebf-8156-318a9ea42694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:17.380928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23319e84-dd67-4ebf-8156-318a9ea42694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:17.382506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23319e84-dd67-4ebf-8156-318a9ea42694 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:17.391748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 696/1425] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:18.299736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d9466-535b-433f-82b5-80b56ac210a8 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:18.300259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d9466-535b-433f-82b5-80b56ac210a8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:18.300351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d9466-535b-433f-82b5-80b56ac210a8 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:18.301726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 730/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:18.306757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d7a5f-cd73-4be0-a1d8-9370259d8486 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:18.307229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d7a5f-cd73-4be0-a1d8-9370259d8486 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:18.307335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d7a5f-cd73-4be0-a1d8-9370259d8486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:18.325786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-106d7a5f-cd73-4be0-a1d8-9370259d8486 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:18.327020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-106d7a5f-cd73-4be0-a1d8-9370259d8486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:18.335434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 697/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:18.485662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d9466-535b-433f-82b5-80b56ac210a8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:18.500182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d9466-535b-433f-82b5-80b56ac210a8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:18.506054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3c9a2-918f-47e7-b9df-b6f1b2d31e30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:18.506685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3c9a2-918f-47e7-b9df-b6f1b2d31e30 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:18.507007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3c9a2-918f-47e7-b9df-b6f1b2d31e30 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:18.507285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3c9a2-918f-47e7-b9df-b6f1b2d31e30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:18.515530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3c9a2-918f-47e7-b9df-b6f1b2d31e30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:18.517139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3c9a2-918f-47e7-b9df-b6f1b2d31e30 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:18.527481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 731/1428] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:19.395113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e69e8b-41c9-4d3a-ae6b-3931652379e1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:19.395691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e69e8b-41c9-4d3a-ae6b-3931652379e1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:19.395961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e69e8b-41c9-4d3a-ae6b-3931652379e1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:19.397734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 698/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:19.403412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01d76-c7bb-465a-b865-def20be571fb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:19.403998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01d76-c7bb-465a-b865-def20be571fb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:19.403998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01d76-c7bb-465a-b865-def20be571fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:19.423677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-63a01d76-c7bb-465a-b865-def20be571fb None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:19.424912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-63a01d76-c7bb-465a-b865-def20be571fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:19.434432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 732/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:19.586545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e69e8b-41c9-4d3a-ae6b-3931652379e1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:19.600960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e69e8b-41c9-4d3a-ae6b-3931652379e1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:19.606243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f7d3e-d06c-4e3b-a9a0-3dbc8a260a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:19.606905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f7d3e-d06c-4e3b-a9a0-3dbc8a260a7a service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:19.607196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f7d3e-d06c-4e3b-a9a0-3dbc8a260a7a service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:19.607477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f7d3e-d06c-4e3b-a9a0-3dbc8a260a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:19.615960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f7d3e-d06c-4e3b-a9a0-3dbc8a260a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:19.617540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f7d3e-d06c-4e3b-a9a0-3dbc8a260a7a service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:19.627131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 699/1431] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:20.479006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771198e3-dc9c-4a9c-92b0-0b6bcdc8febf None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:20.479578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771198e3-dc9c-4a9c-92b0-0b6bcdc8febf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:20.479578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771198e3-dc9c-4a9c-92b0-0b6bcdc8febf None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:20.480954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 733/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:20.485917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663b51b-ae78-4094-801b-1ed460db591c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:20.486362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663b51b-ae78-4094-801b-1ed460db591c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:20.486472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663b51b-ae78-4094-801b-1ed460db591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:20.504175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4663b51b-ae78-4094-801b-1ed460db591c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:20.505950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4663b51b-ae78-4094-801b-1ed460db591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:20.513419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 700/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:20.664779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771198e3-dc9c-4a9c-92b0-0b6bcdc8febf None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:20.683664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771198e3-dc9c-4a9c-92b0-0b6bcdc8febf None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:20.690133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d70a0-e01c-4a32-baa0-46575bfdfec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:20.690775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d70a0-e01c-4a32-baa0-46575bfdfec0 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:20.691090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d70a0-e01c-4a32-baa0-46575bfdfec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:20.691366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d70a0-e01c-4a32-baa0-46575bfdfec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:20.701995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d70a0-e01c-4a32-baa0-46575bfdfec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:20.703671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d70a0-e01c-4a32-baa0-46575bfdfec0 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:20.714066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 734/1434] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:21.581626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e017-a600-4d71-a0fa-563b75590fff None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:21.582193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e017-a600-4d71-a0fa-563b75590fff None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:21.582193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e017-a600-4d71-a0fa-563b75590fff None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:21.584364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 701/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:21.589413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882691ea-43fb-4117-8c60-827ade4f3c0d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:21.589840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882691ea-43fb-4117-8c60-827ade4f3c0d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:21.589977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882691ea-43fb-4117-8c60-827ade4f3c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:21.608382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-882691ea-43fb-4117-8c60-827ade4f3c0d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:21.609572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-882691ea-43fb-4117-8c60-827ade4f3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:21.617306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 735/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:21.768169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9e017-a600-4d71-a0fa-563b75590fff None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:21.782826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9e017-a600-4d71-a0fa-563b75590fff None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:21.788819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb041-32e4-48be-9142-37dcd6bedc29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:21.789448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb041-32e4-48be-9142-37dcd6bedc29 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:21.789733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb041-32e4-48be-9142-37dcd6bedc29 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:21.790013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb041-32e4-48be-9142-37dcd6bedc29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:21.797876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cb041-32e4-48be-9142-37dcd6bedc29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:21.799498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cb041-32e4-48be-9142-37dcd6bedc29 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:21.808944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 702/1437] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:22.661931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eada3-e97b-459c-b005-5fce1c2723c2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:22.662465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eada3-e97b-459c-b005-5fce1c2723c2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:22.662465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eada3-e97b-459c-b005-5fce1c2723c2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:22.663900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 736/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 11:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:22.668938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440aea7-45ed-4c95-a863-d4db69b74804 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:22.669388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440aea7-45ed-4c95-a863-d4db69b74804 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:22.669505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440aea7-45ed-4c95-a863-d4db69b74804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:22.687105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1440aea7-45ed-4c95-a863-d4db69b74804 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:22.688302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1440aea7-45ed-4c95-a863-d4db69b74804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:22.696634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 703/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 11:57:22.844721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eada3-e97b-459c-b005-5fce1c2723c2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:22.858924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eada3-e97b-459c-b005-5fce1c2723c2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:22.864436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b8c0-17e3-4e7c-8734-d62c9de3aec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:22.864921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b8c0-17e3-4e7c-8734-d62c9de3aec7 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:22.865207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b8c0-17e3-4e7c-8734-d62c9de3aec7 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:22.865486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b8c0-17e3-4e7c-8734-d62c9de3aec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:22.873766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9b8c0-17e3-4e7c-8734-d62c9de3aec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:22.875392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9b8c0-17e3-4e7c-8734-d62c9de3aec7 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:22.885322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 737/1440] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:25.119057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47d4f-6315-4a0a-a0d7-ca07d0a902ca None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:25.119463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47d4f-6315-4a0a-a0d7-ca07d0a902ca None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:25.119635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47d4f-6315-4a0a-a0d7-ca07d0a902ca None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:25.121022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 704/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 11:57:25.126441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da23fa-65c3-47bd-985b-9d9a852f6f27 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:25.126909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da23fa-65c3-47bd-985b-9d9a852f6f27 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:25.127019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da23fa-65c3-47bd-985b-9d9a852f6f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:25.144856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-89da23fa-65c3-47bd-985b-9d9a852f6f27 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:25.146916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-89da23fa-65c3-47bd-985b-9d9a852f6f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:25.154638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 738/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:25.193171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47d4f-6315-4a0a-a0d7-ca07d0a902ca None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:25.203813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd47d4f-6315-4a0a-a0d7-ca07d0a902ca None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:25.207281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8114e-f65a-46f3-a487-0406ab38dc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:25.208544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8114e-f65a-46f3-a487-0406ab38dc8e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:25.208928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8114e-f65a-46f3-a487-0406ab38dc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:25.209255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8114e-f65a-46f3-a487-0406ab38dc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:25.217907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8114e-f65a-46f3-a487-0406ab38dc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:25.219377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8114e-f65a-46f3-a487-0406ab38dc8e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:25.228822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 705/1443] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:27.626554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54df818-fba9-46f9-ab67-1d9352554ec1 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:27.627229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54df818-fba9-46f9-ab67-1d9352554ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:27.627229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54df818-fba9-46f9-ab67-1d9352554ec1 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:27.628590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 739/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:27.633958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140700ff-00b6-430d-bf56-7f7290ed0a46 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:27.634399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140700ff-00b6-430d-bf56-7f7290ed0a46 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:27.634523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140700ff-00b6-430d-bf56-7f7290ed0a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:27.652233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-140700ff-00b6-430d-bf56-7f7290ed0a46 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:27.653427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-140700ff-00b6-430d-bf56-7f7290ed0a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:27.661885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 706/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:27.699044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54df818-fba9-46f9-ab67-1d9352554ec1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:27.710144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54df818-fba9-46f9-ab67-1d9352554ec1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:27.713596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd2591-3243-412e-b948-01fc379178e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:27.714221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd2591-3243-412e-b948-01fc379178e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:27.714501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd2591-3243-412e-b948-01fc379178e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:27.714773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd2591-3243-412e-b948-01fc379178e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:27.723500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd2591-3243-412e-b948-01fc379178e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:27.725515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd2591-3243-412e-b948-01fc379178e0 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:27.734807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 740/1446] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 11:57:30.275033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37454e3-beb5-42d2-b9f5-3bf112602305 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:30.275546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37454e3-beb5-42d2-b9f5-3bf112602305 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:30.275546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37454e3-beb5-42d2-b9f5-3bf112602305 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:30.276895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 707/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:30.282225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05000ce9-c309-4dec-895f-479ade2b223e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:30.282617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05000ce9-c309-4dec-895f-479ade2b223e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:30.282787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05000ce9-c309-4dec-895f-479ade2b223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:30.300860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-05000ce9-c309-4dec-895f-479ade2b223e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:30.302691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-05000ce9-c309-4dec-895f-479ade2b223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:30.310383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 741/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:30.348494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37454e3-beb5-42d2-b9f5-3bf112602305 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:30.359782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37454e3-beb5-42d2-b9f5-3bf112602305 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:30.363289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca3939-4b12-45e7-a74d-a731cfa13e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:30.363904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3939-4b12-45e7-a74d-a731cfa13e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:30.364188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3939-4b12-45e7-a74d-a731cfa13e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:30.364465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca3939-4b12-45e7-a74d-a731cfa13e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:30.374071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca3939-4b12-45e7-a74d-a731cfa13e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:30.375536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca3939-4b12-45e7-a74d-a731cfa13e8e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:30.384763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 708/1449] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:32.764682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692236-fb21-4bee-9a30-a3aec3e622d4 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:32.765248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692236-fb21-4bee-9a30-a3aec3e622d4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:32.765353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692236-fb21-4bee-9a30-a3aec3e622d4 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:32.766702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 742/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:32.772793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc885-74bd-469e-87e1-efcabb79be96 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:32.773222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc885-74bd-469e-87e1-efcabb79be96 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:32.773347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cc885-74bd-469e-87e1-efcabb79be96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:32.790476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-df4cc885-74bd-469e-87e1-efcabb79be96 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:32.791803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-df4cc885-74bd-469e-87e1-efcabb79be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:32.799516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 709/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:32.839060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692236-fb21-4bee-9a30-a3aec3e622d4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:32.849944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692236-fb21-4bee-9a30-a3aec3e622d4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:32.853449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360357a6-eacd-43da-96c1-ba2016e8f2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:32.854083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360357a6-eacd-43da-96c1-ba2016e8f2db service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:32.854367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360357a6-eacd-43da-96c1-ba2016e8f2db service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:32.854647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360357a6-eacd-43da-96c1-ba2016e8f2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:32.863487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360357a6-eacd-43da-96c1-ba2016e8f2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:32.865587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360357a6-eacd-43da-96c1-ba2016e8f2db service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:32.875238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 743/1452] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:35.241922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f096b1-ced4-4fcf-80b4-bd26a3ada670 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:35.242484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f096b1-ced4-4fcf-80b4-bd26a3ada670 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:35.242609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f096b1-ced4-4fcf-80b4-bd26a3ada670 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:35.243953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 710/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:35.249730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6de018-13c0-4944-be1f-3e9cea97e32e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:35.250148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6de018-13c0-4944-be1f-3e9cea97e32e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:35.250294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6de018-13c0-4944-be1f-3e9cea97e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:35.268093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0d6de018-13c0-4944-be1f-3e9cea97e32e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:35.269214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0d6de018-13c0-4944-be1f-3e9cea97e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:35.277591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 744/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:35.315636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f096b1-ced4-4fcf-80b4-bd26a3ada670 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:35.326386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f096b1-ced4-4fcf-80b4-bd26a3ada670 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:35.329836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b206-717e-468d-80ab-2381caf9eb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:35.330493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b206-717e-468d-80ab-2381caf9eb12 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:35.330776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b206-717e-468d-80ab-2381caf9eb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:35.331084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b206-717e-468d-80ab-2381caf9eb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:35.340673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87b206-717e-468d-80ab-2381caf9eb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:35.342168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87b206-717e-468d-80ab-2381caf9eb12 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:35.352424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 711/1455] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:37.649334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ee75-2e0d-431f-9ccd-9d72ae6b57e2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:37.649902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ee75-2e0d-431f-9ccd-9d72ae6b57e2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:37.649902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9ee75-2e0d-431f-9ccd-9d72ae6b57e2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:37.651297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 745/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:37.656498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64d07d-dbec-4599-98ca-5996f885b103 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:37.656924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64d07d-dbec-4599-98ca-5996f885b103 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:37.657069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64d07d-dbec-4599-98ca-5996f885b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:37.676088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bf64d07d-dbec-4599-98ca-5996f885b103 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:37.677283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bf64d07d-dbec-4599-98ca-5996f885b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:37.685149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 712/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:37.723375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9ee75-2e0d-431f-9ccd-9d72ae6b57e2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:37.735739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9ee75-2e0d-431f-9ccd-9d72ae6b57e2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:37.739366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f292ca-092e-4485-82e0-636ed49db5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:37.740027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f292ca-092e-4485-82e0-636ed49db5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:37.740849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f292ca-092e-4485-82e0-636ed49db5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:37.741127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f292ca-092e-4485-82e0-636ed49db5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:37.750158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f292ca-092e-4485-82e0-636ed49db5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:37.751607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f292ca-092e-4485-82e0-636ed49db5e5 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:37.760812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 746/1458] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:40.054347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e819105-fcc9-4c67-a0fe-578e63e358e2 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:40.054870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e819105-fcc9-4c67-a0fe-578e63e358e2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:40.054979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e819105-fcc9-4c67-a0fe-578e63e358e2 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:40.056391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 713/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:40.061572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dd56-c2a1-4303-82fc-4e7607abed12 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:40.061980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dd56-c2a1-4303-82fc-4e7607abed12 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:40.062134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dd56-c2a1-4303-82fc-4e7607abed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:40.068143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=70 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:57:40.081151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cbe5dd56-c2a1-4303-82fc-4e7607abed12 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:40.082344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cbe5dd56-c2a1-4303-82fc-4e7607abed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:40.090721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 747/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:40.128328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e819105-fcc9-4c67-a0fe-578e63e358e2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:40.130045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1e819105-fcc9-4c67-a0fe-578e63e358e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:40.138428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=73 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:57:40.147912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e819105-fcc9-4c67-a0fe-578e63e358e2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:40.151614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd63f65-621e-4150-90f8-e247b8127e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:40.152288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd63f65-621e-4150-90f8-e247b8127e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:40.152572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd63f65-621e-4150-90f8-e247b8127e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:40.152850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd63f65-621e-4150-90f8-e247b8127e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:40.162150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd63f65-621e-4150-90f8-e247b8127e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:40.163609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd63f65-621e-4150-90f8-e247b8127e93 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:40.172797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 714/1461] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:42.477743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43742c4-37fc-4408-bd7a-63ceaf65f787 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:42.478311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43742c4-37fc-4408-bd7a-63ceaf65f787 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:42.478311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43742c4-37fc-4408-bd7a-63ceaf65f787 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:42.479714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 748/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:42.485136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc1eca-6351-4fea-9fbe-254090694e17 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:42.485553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc1eca-6351-4fea-9fbe-254090694e17 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:42.485696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc1eca-6351-4fea-9fbe-254090694e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:42.504029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-65dc1eca-6351-4fea-9fbe-254090694e17 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:42.505219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-65dc1eca-6351-4fea-9fbe-254090694e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:42.513020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 715/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:42.557427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43742c4-37fc-4408-bd7a-63ceaf65f787 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:42.568113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43742c4-37fc-4408-bd7a-63ceaf65f787 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:42.571702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391650c0-0d01-48d8-aedc-bf1e338ae9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:42.572336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391650c0-0d01-48d8-aedc-bf1e338ae9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:42.572623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391650c0-0d01-48d8-aedc-bf1e338ae9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:42.572906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391650c0-0d01-48d8-aedc-bf1e338ae9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:42.581790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391650c0-0d01-48d8-aedc-bf1e338ae9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:42.583271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391650c0-0d01-48d8-aedc-bf1e338ae9e3 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:42.593295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 749/1464] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:44.907269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e885e-eecd-474b-b69b-26cc4fb602fb None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:44.907817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e885e-eecd-474b-b69b-26cc4fb602fb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:44.907817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e885e-eecd-474b-b69b-26cc4fb602fb None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:44.909214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 716/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:44.915073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a680c71-34f0-4d27-b8f4-55ac8dfc117f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:44.915524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a680c71-34f0-4d27-b8f4-55ac8dfc117f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:44.915645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a680c71-34f0-4d27-b8f4-55ac8dfc117f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:44.933070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1a680c71-34f0-4d27-b8f4-55ac8dfc117f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:44.934278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1a680c71-34f0-4d27-b8f4-55ac8dfc117f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:44.942029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 750/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:44.979436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e885e-eecd-474b-b69b-26cc4fb602fb None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:44.989895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e885e-eecd-474b-b69b-26cc4fb602fb None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:44.993546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81399a08-6b74-48fe-8913-994f6cf33b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:44.994182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81399a08-6b74-48fe-8913-994f6cf33b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:44.994469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81399a08-6b74-48fe-8913-994f6cf33b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:44.994750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81399a08-6b74-48fe-8913-994f6cf33b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:45.003881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81399a08-6b74-48fe-8913-994f6cf33b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:45.005322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81399a08-6b74-48fe-8913-994f6cf33b79 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:45.015084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 717/1467] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:47.282210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152a4e4-4024-4655-9fec-d90fe03cae3f None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:47.282732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152a4e4-4024-4655-9fec-d90fe03cae3f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:47.282732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152a4e4-4024-4655-9fec-d90fe03cae3f None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:47.284176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 751/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 11:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:57:47.289715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d625fb-0eda-4206-b95e-f123b005443e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:47.290157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d625fb-0eda-4206-b95e-f123b005443e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:47.290266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d625fb-0eda-4206-b95e-f123b005443e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:47.308567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-61d625fb-0eda-4206-b95e-f123b005443e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:57:47.309755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-61d625fb-0eda-4206-b95e-f123b005443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:57:47.317377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 718/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 11:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:57:47.354728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152a4e4-4024-4655-9fec-d90fe03cae3f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:57:47.365330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152a4e4-4024-4655-9fec-d90fe03cae3f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:57:47.369364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b06d-d877-499c-b199-f568562b54e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:57:47.370278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b06d-d877-499c-b199-f568562b54e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:57:47.370574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b06d-d877-499c-b199-f568562b54e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:57:47.370877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b06d-d877-499c-b199-f568562b54e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:57:47.379949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b06d-d877-499c-b199-f568562b54e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:57:47.381399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b06d-d877-499c-b199-f568562b54e2 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:57:47.391269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 752/1470] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:57:57.302445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=17 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:57:57.387108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=17 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:58:19.318832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f77ab-71ff-4677-9374-60dc56c661bf None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:19.319431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f77ab-71ff-4677-9374-60dc56c661bf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:19.319431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f77ab-71ff-4677-9374-60dc56c661bf None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:19.320784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 719/1471] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:19.326037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dbf6e-ecc1-450c-b750-cab4c843a844 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:19.326489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dbf6e-ecc1-450c-b750-cab4c843a844 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:19.326598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dbf6e-ecc1-450c-b750-cab4c843a844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:19.347841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d57dbf6e-ecc1-450c-b750-cab4c843a844 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:19.349038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d57dbf6e-ecc1-450c-b750-cab4c843a844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:19.356751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 753/1472] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:58:19.403505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ab8d-88c6-43bd-ba16-8a888114c990 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:19.403797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ab8d-88c6-43bd-ba16-8a888114c990 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:19.404074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ab8d-88c6-43bd-ba16-8a888114c990 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:19.405429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 720/1473] 10.222.0.41 () {54 vars in 962 bytes} [Mon Jun 6 11:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:19.410314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3297cd-1ebc-425f-99b6-d130885beb0a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:19.410605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3297cd-1ebc-425f-99b6-d130885beb0a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:19.410910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3297cd-1ebc-425f-99b6-d130885beb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:19.428976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6c3297cd-1ebc-425f-99b6-d130885beb0a None None] MFA Rules not processed for user `d0fa7ced305e4d2fa0d791675f839453`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:19.430160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6c3297cd-1ebc-425f-99b6-d130885beb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:19.436472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 754/1474] 10.222.0.41 () {58 vars in 1059 bytes} [Mon Jun 6 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:58:19.444708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ab8d-88c6-43bd-ba16-8a888114c990 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:19.455343 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ab8d-88c6-43bd-ba16-8a888114c990 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:19.459268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b7b8-8c7a-4c57-bd2b-bf8c6bab44d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:19.459889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b7b8-8c7a-4c57-bd2b-bf8c6bab44d9 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:19.460176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b7b8-8c7a-4c57-bd2b-bf8c6bab44d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:19.460453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b7b8-8c7a-4c57-bd2b-bf8c6bab44d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:19.469005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b7b8-8c7a-4c57-bd2b-bf8c6bab44d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:19.470415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b7b8-8c7a-4c57-bd2b-bf8c6bab44d9 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:19.480397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 721/1475] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:58:21.826628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30f71b-bf3a-4137-9f1a-83bf70545251 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:21.827294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30f71b-bf3a-4137-9f1a-83bf70545251 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:21.827294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30f71b-bf3a-4137-9f1a-83bf70545251 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:21.828665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 755/1476] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:21.834061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8721-4ef3-48e4-83dd-4b8bc24280a8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:21.834467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8721-4ef3-48e4-83dd-4b8bc24280a8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:21.834609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc8721-4ef3-48e4-83dd-4b8bc24280a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:21.852597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b5dc8721-4ef3-48e4-83dd-4b8bc24280a8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:21.853782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b5dc8721-4ef3-48e4-83dd-4b8bc24280a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:21.862010 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 722/1477] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:58:21.885646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30f71b-bf3a-4137-9f1a-83bf70545251 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:21.898042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30f71b-bf3a-4137-9f1a-83bf70545251 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:21.902434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:21.903111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:21.903399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:21.903677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:21.904495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:21.906232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 11:58:21.906232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 11:58:21.906232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 11:58:21.907447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1935220-a84d-4366-872a-47bd18a83bee admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:21.928828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 756/1478] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 11:58:21 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 11:58:24.213307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a495831-ea28-4a07-933e-49f23ce104b1 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:24.213836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a495831-ea28-4a07-933e-49f23ce104b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:24.213836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a495831-ea28-4a07-933e-49f23ce104b1 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:24.215329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 723/1479] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:24.221472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c34dd2-cdee-4805-8566-0f323ca7c74d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:24.221895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c34dd2-cdee-4805-8566-0f323ca7c74d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:24.222054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c34dd2-cdee-4805-8566-0f323ca7c74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:24.240933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b3c34dd2-cdee-4805-8566-0f323ca7c74d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:24.242149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b3c34dd2-cdee-4805-8566-0f323ca7c74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:24.250242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 757/1480] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 11:58:24.294617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a495831-ea28-4a07-933e-49f23ce104b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:24.309997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a495831-ea28-4a07-933e-49f23ce104b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:24.315802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9af223-4bd0-4f88-b7e2-bdc82730732a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:24.316459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9af223-4bd0-4f88-b7e2-bdc82730732a service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:24.316747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9af223-4bd0-4f88-b7e2-bdc82730732a service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:24.317028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9af223-4bd0-4f88-b7e2-bdc82730732a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:24.325810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9af223-4bd0-4f88-b7e2-bdc82730732a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:24.327923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9af223-4bd0-4f88-b7e2-bdc82730732a service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:24.337760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 724/1481] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:58:26.837912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b132094-fe34-45ea-a1fd-11c109214603 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:26.838484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b132094-fe34-45ea-a1fd-11c109214603 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:26.838484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b132094-fe34-45ea-a1fd-11c109214603 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:26.839922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 758/1482] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:26.845682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e461-20d3-4236-b1e5-97f5ba23c636 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:26.846126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e461-20d3-4236-b1e5-97f5ba23c636 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:26.846234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e461-20d3-4236-b1e5-97f5ba23c636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:26.865577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1f14e461-20d3-4236-b1e5-97f5ba23c636 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:26.866768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1f14e461-20d3-4236-b1e5-97f5ba23c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:26.874611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 725/1483] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:58:26.912381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b132094-fe34-45ea-a1fd-11c109214603 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:26.924068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b132094-fe34-45ea-a1fd-11c109214603 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:26.927637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a132d-063e-45db-a7aa-898e130c0308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:26.928278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a132d-063e-45db-a7aa-898e130c0308 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:26.928560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a132d-063e-45db-a7aa-898e130c0308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:26.928838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a132d-063e-45db-a7aa-898e130c0308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:26.937830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a132d-063e-45db-a7aa-898e130c0308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:26.939307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a132d-063e-45db-a7aa-898e130c0308 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:26.949606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 759/1484] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:58:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:58:29.258175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4c20-9d7f-4e1c-b5d8-86e5d39ce335 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:29.258732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4c20-9d7f-4e1c-b5d8-86e5d39ce335 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:29.258732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4c20-9d7f-4e1c-b5d8-86e5d39ce335 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:29.260125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 726/1485] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:29.266162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3526f-a43b-4983-9965-032f8a5e4fbe None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:29.266593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3526f-a43b-4983-9965-032f8a5e4fbe None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:29.266765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3526f-a43b-4983-9965-032f8a5e4fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:29.284215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d0b3526f-a43b-4983-9965-032f8a5e4fbe None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:29.285403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d0b3526f-a43b-4983-9965-032f8a5e4fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:29.293049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 760/1486] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 11:58:29.333176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e4c20-9d7f-4e1c-b5d8-86e5d39ce335 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:29.344609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e4c20-9d7f-4e1c-b5d8-86e5d39ce335 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:29.348199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32013d7-54a9-49de-97ce-72c9033a33e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:29.348826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32013d7-54a9-49de-97ce-72c9033a33e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:29.349106 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32013d7-54a9-49de-97ce-72c9033a33e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:29.349380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32013d7-54a9-49de-97ce-72c9033a33e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:29.357983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32013d7-54a9-49de-97ce-72c9033a33e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:29.359507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32013d7-54a9-49de-97ce-72c9033a33e8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:29.368485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 727/1487] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:58:31.713831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe08ed-8ca8-4f60-a847-d4e96fab18d2 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:31.714407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe08ed-8ca8-4f60-a847-d4e96fab18d2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:31.714407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe08ed-8ca8-4f60-a847-d4e96fab18d2 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:31.715817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 761/1488] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:31.721700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc5e4f-523a-4c62-addd-853fb377bf4d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:31.722118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc5e4f-523a-4c62-addd-853fb377bf4d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:31.722270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc5e4f-523a-4c62-addd-853fb377bf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:31.739672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6ecc5e4f-523a-4c62-addd-853fb377bf4d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:31.740856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6ecc5e4f-523a-4c62-addd-853fb377bf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:31.748869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 728/1489] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:58:31.790950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe08ed-8ca8-4f60-a847-d4e96fab18d2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:31.801955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe08ed-8ca8-4f60-a847-d4e96fab18d2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:31.805477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd38886-6730-4036-a14d-07a0575b39ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:31.806112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd38886-6730-4036-a14d-07a0575b39ac service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:31.806400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd38886-6730-4036-a14d-07a0575b39ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:31.806680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd38886-6730-4036-a14d-07a0575b39ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:31.815754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd38886-6730-4036-a14d-07a0575b39ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:31.817195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd38886-6730-4036-a14d-07a0575b39ac service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:31.827092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 762/1490] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:58:34.337515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef939f-74bb-44dd-9d98-c79273442775 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:34.338101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef939f-74bb-44dd-9d98-c79273442775 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:34.338101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef939f-74bb-44dd-9d98-c79273442775 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:34.339464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 729/1491] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 11:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 11:58:34.344942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1d001-8e3c-4667-b525-65dc0389b928 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:34.345373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1d001-8e3c-4667-b525-65dc0389b928 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:34.345517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1d001-8e3c-4667-b525-65dc0389b928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:34.363163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a3a1d001-8e3c-4667-b525-65dc0389b928 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 11:58:34.364369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a3a1d001-8e3c-4667-b525-65dc0389b928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 11:58:34.373100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 763/1492] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 11:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 11:58:34.411237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef939f-74bb-44dd-9d98-c79273442775 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 11:58:34.421688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef939f-74bb-44dd-9d98-c79273442775 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 11:58:34.425118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb400552-7ded-4df8-934b-18ecbd27210c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 11:58:34.425740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb400552-7ded-4df8-934b-18ecbd27210c service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 11:58:34.426021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb400552-7ded-4df8-934b-18ecbd27210c service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 11:58:34.426295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb400552-7ded-4df8-934b-18ecbd27210c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 11:58:34.435496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb400552-7ded-4df8-934b-18ecbd27210c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 11:58:34.436910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb400552-7ded-4df8-934b-18ecbd27210c service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 11:58:34.445949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 730/1493] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 11:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 11:58:44.357977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=33 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 11:58:44.441918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=33 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:00:00.493537 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.494155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.494155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.514339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:00:00.515570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:00.524109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 764/1494] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:00:00.536674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.537199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.537303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.558777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:00:00.560015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:00.577409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 731/1495] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:00:00.583282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:00.596214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589bb3d-e18d-45a2-8a34-95fa94cdc0ca None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:00.601151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:00.601807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.602091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.602370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.603114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:00:00.603422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:00.604438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:00.604438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:00:00.604438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:00:00.605323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:00.617453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 765/1496] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:00:00.639366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:00.649414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a3cc8-8fcd-46fa-ba3d-83be79f713a8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:00.660096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:00.660815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.661092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.661368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.662058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:00.663863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:00.663863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:00:00.663863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:00:00.665054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:00.728390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 732/1497] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:00:00 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:00:00.733053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:00.746930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc13d-0a07-435a-9524-619e8aaa6de5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:00.809698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:00.810374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.810664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.810979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.811695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:00.812724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:00.812724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:00:00.812724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:00:00.813950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:00.835198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 766/1498] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:00:00 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:00:00.839460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:00.850193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aad204-688b-4b62-bae5-ad8d452e1429 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:00.854749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:00.855443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.855745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.856014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.856734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:00.857755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:00.857755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:00:00.857755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:00:00.858640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:00.872187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 733/1499] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:00:00.876486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:00.887595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef67a8-ae9c-40b1-8825-9486209f29e0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:00.892299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:00.892965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.893258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.893541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.894270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:00.895796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:00.895796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:00:00.895796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:00:00.897429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d02c97-e32e-4d41-b204-5e86972d53b3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:00.912515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 767/1500] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:00:00.917127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:00.928066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d2b7-da27-4da2-8bd5-d96dba6e4eac admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:00.932669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:00.933568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.933854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.934136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] PATH_INFO: `/v3/projects/3ab0e463f6774548b10d3d6a918eed71/users/9baae67c9bdb4ae7a790d8e5480a4a7e/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:00.950406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab0e463f6774548b10d3d6a918eed71, user_id=9baae67c9bdb4ae7a790d8e5480a4a7e, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:00.951491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:00.951491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:00:00.951491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:00:00.953409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b05879-0804-47f3-9b81-3bc92e82e905 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:00.966355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 734/1501] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:00:00 2022] PUT /identity/v3/projects/3ab0e463f6774548b10d3d6a918eed71/users/9baae67c9bdb4ae7a790d8e5480a4a7e/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:00:00.980013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3b31-6d74-4765-829d-e8238ab2e054 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:00.980304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3b31-6d74-4765-829d-e8238ab2e054 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:00.980586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3b31-6d74-4765-829d-e8238ab2e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.008257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-170b3b31-6d74-4765-829d-e8238ab2e054 None None] MFA Rules not processed for user `9baae67c9bdb4ae7a790d8e5480a4a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:00:01.009414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-170b3b31-6d74-4765-829d-e8238ab2e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:01.080301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 768/1502] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:00:01.093515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b475191-2bbd-40a6-98da-4f4895e2e4fe None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.093808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b475191-2bbd-40a6-98da-4f4895e2e4fe None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.094093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b475191-2bbd-40a6-98da-4f4895e2e4fe None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.095520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 735/1503] 10.222.0.41 () {54 vars in 963 bytes} [Mon Jun 6 12:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:00:01.101348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c699bf-de75-49dc-be7d-28945e241644 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.101646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c699bf-de75-49dc-be7d-28945e241644 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.101937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c699bf-de75-49dc-be7d-28945e241644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.121159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-25c699bf-de75-49dc-be7d-28945e241644 None None] MFA Rules not processed for user `106467a9f0644177872555b7998599d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:00:01.122430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-25c699bf-de75-49dc-be7d-28945e241644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:01.192937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:00:01.201358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 769/1504] 10.222.0.41 () {58 vars in 1060 bytes} [Mon Jun 6 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:00:01.209425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b475191-2bbd-40a6-98da-4f4895e2e4fe None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:01.223447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b475191-2bbd-40a6-98da-4f4895e2e4fe None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:01.227170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:01.227814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.228107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.228387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.237694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:01.240144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:01.249211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 736/1505] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:00:01.294360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812f77-a237-4369-ab5b-83f959bf457e None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.294659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812f77-a237-4369-ab5b-83f959bf457e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.294989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812f77-a237-4369-ab5b-83f959bf457e None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.296300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 770/1506] 10.222.0.41 () {52 vars in 829 bytes} [Mon Jun 6 12:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:00:01.302259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:01.377724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96968b37-f8b7-42ff-965b-8c1fec137c12 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:01.383327 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:01.384472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.384776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.385047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] PATH_INFO: `/v3/users/9baae67c9bdb4ae7a790d8e5480a4a7e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.386361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9baae67c9bdb4ae7a790d8e5480a4a7e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:01.387432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:01.387432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:00:01.387432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:00:01.388763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:01.462924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9baae67c9bdb4ae7a790d8e5480a4a7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:01.463662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9baae67c9bdb4ae7a790d8e5480a4a7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:01.464279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9baae67c9bdb4ae7a790d8e5480a4a7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:01.478504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9baae67c9bdb4ae7a790d8e5480a4a7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:01.497598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9baae67c9bdb4ae7a790d8e5480a4a7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:01.501694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:00:01.515655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 737/1507] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:00:01 2022] DELETE /identity/v3/users/9baae67c9bdb4ae7a790d8e5480a4a7e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:00:01.520968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.521436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.521436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.539210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:00:01.540423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:01.600338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 771/1508] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:00:01.610684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:01.612107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:01.679774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c854c19-e8e1-4127-ae1a-ecb2d5968548 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:01.683341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7f939-3156-43f3-9e29-c62f06cfe97a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:01.684079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f939-3156-43f3-9e29-c62f06cfe97a service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:01.684363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f939-3156-43f3-9e29-c62f06cfe97a service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:01.684642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f939-3156-43f3-9e29-c62f06cfe97a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:01.698354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7f939-3156-43f3-9e29-c62f06cfe97a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:01.699849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7f939-3156-43f3-9e29-c62f06cfe97a service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:01.710394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 738/1509] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:00:02.182569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:00:02.184201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:00:02.198512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4fbde-b98f-4260-a583-86490dd7e29e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:00:02.204841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:00:02.205714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:00:02.207031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:00:02.207369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] PATH_INFO: `/v3/projects/3ab0e463f6774548b10d3d6a918eed71` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:00:02.209277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab0e463f6774548b10d3d6a918eed71)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:00:02.210627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:00:02.210627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:00:02.210627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:00:02.213218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:00:02.276286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab0e463f6774548b10d3d6a918eed71', 'request_id': 'req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:02.289640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab0e463f6774548b10d3d6a918eed71', 'request_id': 'req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:02.299413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab0e463f6774548b10d3d6a918eed71', 'request_id': 'req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:02.310989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] The token cache is being invalidate because project 3ab0e463f6774548b10d3d6a918eed71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:00:02.311278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:02.311911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ba8abc05-26f0-41d2-9b2b-674a3ae9a61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:00:02.313530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 772/1510] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:00:02 2022] DELETE /identity/v3/projects/3ab0e463f6774548b10d3d6a918eed71 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:00:11.705377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:00:12.308017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:01:36.991617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396a90a-38a8-4150-8161-145e9c734eee None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:36.992216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396a90a-38a8-4150-8161-145e9c734eee None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:36.992216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396a90a-38a8-4150-8161-145e9c734eee None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:36.993546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 739/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 12:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:36.998691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef59068-6173-4a6c-8b26-90fef4e5ffa7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:36.999168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef59068-6173-4a6c-8b26-90fef4e5ffa7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:36.999274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef59068-6173-4a6c-8b26-90fef4e5ffa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:37.017637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-aef59068-6173-4a6c-8b26-90fef4e5ffa7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:37.018870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-aef59068-6173-4a6c-8b26-90fef4e5ffa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:37.077404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 773/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:01:37.249943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396a90a-38a8-4150-8161-145e9c734eee None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:37.254413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6396a90a-38a8-4150-8161-145e9c734eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:37.328167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396a90a-38a8-4150-8161-145e9c734eee None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:37.333475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd3137-91e8-40d8-ac58-90ba350de1f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:37.334112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd3137-91e8-40d8-ac58-90ba350de1f3 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:37.334397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd3137-91e8-40d8-ac58-90ba350de1f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:37.334675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd3137-91e8-40d8-ac58-90ba350de1f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:37.342913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd3137-91e8-40d8-ac58-90ba350de1f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:37.344499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd3137-91e8-40d8-ac58-90ba350de1f3 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:37.353610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 740/1513] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:01:38.255481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef4f7-d39e-4d6f-9b18-62195c05ba39 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:38.256010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef4f7-d39e-4d6f-9b18-62195c05ba39 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:38.256010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef4f7-d39e-4d6f-9b18-62195c05ba39 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:38.257376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 774/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 12:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:38.262238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:38.262694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:38.262828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:38.281414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:38.282597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:38.290450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 741/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:01:38.443551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88328c59-1b3e-4140-9f39-d300d274a013 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:38.443789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:38.443892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88328c59-1b3e-4140-9f39-d300d274a013 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:38.444118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88328c59-1b3e-4140-9f39-d300d274a013 None None] PATH_INFO: `/v3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:38.445428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 775/1516] 10.222.0.44 () {54 vars in 925 bytes} [Mon Jun 6 12:01:38 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:01:38.451805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45250bb-4b53-49b8-98d3-7e937774e45f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:38.452099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45250bb-4b53-49b8-98d3-7e937774e45f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:38.452382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45250bb-4b53-49b8-98d3-7e937774e45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:38.456232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62d1f8-bbaf-4a4b-8cb7-d67a40f6b5e8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:38.461609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4abd3-11c1-4116-8447-180d17729606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:38.462253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4abd3-11c1-4116-8447-180d17729606 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:38.462548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4abd3-11c1-4116-8447-180d17729606 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:38.463107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4abd3-11c1-4116-8447-180d17729606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:38.471146 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e45250bb-4b53-49b8-98d3-7e937774e45f None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:38.472302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4abd3-11c1-4116-8447-180d17729606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:38.472439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e45250bb-4b53-49b8-98d3-7e937774e45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:38.473876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4abd3-11c1-4116-8447-180d17729606 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:38.484348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 742/1517] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:01:38.548956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 776/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Mon Jun 6 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:01:47.711995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca95fd-ed45-4f79-9d27-85ccfa0bb5fa None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:47.712800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca95fd-ed45-4f79-9d27-85ccfa0bb5fa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:47.712800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca95fd-ed45-4f79-9d27-85ccfa0bb5fa None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:47.714186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 743/1519] 10.222.0.41 () {54 vars in 960 bytes} [Mon Jun 6 12:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:47.720291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2a29-c958-427f-a06e-aa6478d1ea27 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:47.720719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2a29-c958-427f-a06e-aa6478d1ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:47.720861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2a29-c958-427f-a06e-aa6478d1ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:47.738650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cf5d2a29-c958-427f-a06e-aa6478d1ea27 None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:47.739886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cf5d2a29-c958-427f-a06e-aa6478d1ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:47.749034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 777/1520] 10.222.0.41 () {58 vars in 1057 bytes} [Mon Jun 6 12:01:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:01:47.756351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca95fd-ed45-4f79-9d27-85ccfa0bb5fa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:47.769827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca95fd-ed45-4f79-9d27-85ccfa0bb5fa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:47.774822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0920140-e4ca-4c5e-81b2-4989050ca09f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:47.775485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920140-e4ca-4c5e-81b2-4989050ca09f service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:47.775772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920140-e4ca-4c5e-81b2-4989050ca09f service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:47.776055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0920140-e4ca-4c5e-81b2-4989050ca09f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:47.786137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0920140-e4ca-4c5e-81b2-4989050ca09f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:47.788058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0920140-e4ca-4c5e-81b2-4989050ca09f service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:47.798798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 744/1521] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:01:49.762550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96660f6-aabf-4b70-b9f5-f7ffbf35e207 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:49.763185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96660f6-aabf-4b70-b9f5-f7ffbf35e207 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:49.763185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96660f6-aabf-4b70-b9f5-f7ffbf35e207 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:49.764523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 778/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 12:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:49.769307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309f357-34ee-458f-b6b8-2c47acfbdd8e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:49.769688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309f357-34ee-458f-b6b8-2c47acfbdd8e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:49.769868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309f357-34ee-458f-b6b8-2c47acfbdd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:49.788332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d309f357-34ee-458f-b6b8-2c47acfbdd8e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:49.789527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d309f357-34ee-458f-b6b8-2c47acfbdd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:49.797795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 745/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 12:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:01:49.954833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96660f6-aabf-4b70-b9f5-f7ffbf35e207 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:49.970197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96660f6-aabf-4b70-b9f5-f7ffbf35e207 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:49.975574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7580c-e5a2-48c4-92d1-e5788f0d9666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:49.976215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7580c-e5a2-48c4-92d1-e5788f0d9666 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:49.976505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7580c-e5a2-48c4-92d1-e5788f0d9666 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:49.976788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7580c-e5a2-48c4-92d1-e5788f0d9666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:49.986020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7580c-e5a2-48c4-92d1-e5788f0d9666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:49.987670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7580c-e5a2-48c4-92d1-e5788f0d9666 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:49.997167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 779/1524] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:01:50.873500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3af3-b71d-403b-9e6e-e01a3e26c7ed None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:50.874060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3af3-b71d-403b-9e6e-e01a3e26c7ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:50.874060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3af3-b71d-403b-9e6e-e01a3e26c7ed None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:50.875508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 746/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 12:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:50.880485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd3360-2d7a-4ae5-a386-47f14d31b18e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:50.880936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd3360-2d7a-4ae5-a386-47f14d31b18e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:50.881036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd3360-2d7a-4ae5-a386-47f14d31b18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:50.898947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-adfd3360-2d7a-4ae5-a386-47f14d31b18e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:50.900150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-adfd3360-2d7a-4ae5-a386-47f14d31b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:50.908037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 780/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 12:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:01:51.060109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3af3-b71d-403b-9e6e-e01a3e26c7ed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:51.074362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3af3-b71d-403b-9e6e-e01a3e26c7ed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:51.079758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea41a6-b32f-4bc2-a02a-77ecbec332e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:51.080394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea41a6-b32f-4bc2-a02a-77ecbec332e0 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:51.080684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea41a6-b32f-4bc2-a02a-77ecbec332e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:51.080970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea41a6-b32f-4bc2-a02a-77ecbec332e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:51.089221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea41a6-b32f-4bc2-a02a-77ecbec332e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:51.091481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea41a6-b32f-4bc2-a02a-77ecbec332e0 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:51.100725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 747/1527] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:01:55.063943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86d7c8-ba1a-4d42-9836-1d91e55a8e36 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:55.064476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86d7c8-ba1a-4d42-9836-1d91e55a8e36 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:55.064586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86d7c8-ba1a-4d42-9836-1d91e55a8e36 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:55.065889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 781/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 12:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:55.071122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1626aa1-8ed5-4ce6-bbe8-359d48197444 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:55.071565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1626aa1-8ed5-4ce6-bbe8-359d48197444 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:55.071675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1626aa1-8ed5-4ce6-bbe8-359d48197444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:55.090872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f1626aa1-8ed5-4ce6-bbe8-359d48197444 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:55.092250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f1626aa1-8ed5-4ce6-bbe8-359d48197444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:55.099994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 748/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 12:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:01:55.136637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86d7c8-ba1a-4d42-9836-1d91e55a8e36 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:55.138967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9c86d7c8-ba1a-4d42-9836-1d91e55a8e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:55.195974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86d7c8-ba1a-4d42-9836-1d91e55a8e36 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:55.200283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93344bec-8392-4f90-bce2-4fb33f92778a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:55.200957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93344bec-8392-4f90-bce2-4fb33f92778a service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:55.201245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93344bec-8392-4f90-bce2-4fb33f92778a service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:55.201525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93344bec-8392-4f90-bce2-4fb33f92778a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:55.210530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93344bec-8392-4f90-bce2-4fb33f92778a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:55.212013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93344bec-8392-4f90-bce2-4fb33f92778a service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:55.221839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 782/1530] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:01:57.448890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1eb075-e2fe-43f8-b377-afb6c5bd49da None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:57.449504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1eb075-e2fe-43f8-b377-afb6c5bd49da None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:57.449504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1eb075-e2fe-43f8-b377-afb6c5bd49da None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:57.451338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 749/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 12:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:01:57.456513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42f69-4d61-40f8-a043-ff85322df3ca None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:57.456961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42f69-4d61-40f8-a043-ff85322df3ca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:57.457058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42f69-4d61-40f8-a043-ff85322df3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:57.476129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1ef42f69-4d61-40f8-a043-ff85322df3ca None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:01:57.477324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1ef42f69-4d61-40f8-a043-ff85322df3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:01:57.485939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 783/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 12:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:01:57.521403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1eb075-e2fe-43f8-b377-afb6c5bd49da None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:01:57.532204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1eb075-e2fe-43f8-b377-afb6c5bd49da None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:01:57.535633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b3191-c6d0-48f4-98d7-87dade771581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:01:57.536260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3191-c6d0-48f4-98d7-87dade771581 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:01:57.536543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3191-c6d0-48f4-98d7-87dade771581 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:01:57.536825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3191-c6d0-48f4-98d7-87dade771581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:01:57.546496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b3191-c6d0-48f4-98d7-87dade771581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:01:57.547948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b3191-c6d0-48f4-98d7-87dade771581 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:01:57.557094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 750/1533] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:02:01.602964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9929c-84e9-4917-931e-de26e5816f22 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:02:01.603508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9929c-84e9-4917-931e-de26e5816f22 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:02:01.603508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9929c-84e9-4917-931e-de26e5816f22 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:02:01.604906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 784/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 12:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:02:01.610309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5274b-e9a4-4d99-9408-14f71457b977 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:02:01.610757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5274b-e9a4-4d99-9408-14f71457b977 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:02:01.610894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5274b-e9a4-4d99-9408-14f71457b977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:02:01.629338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-43b5274b-e9a4-4d99-9408-14f71457b977 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:02:01.630521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-43b5274b-e9a4-4d99-9408-14f71457b977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:02:01.639126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 751/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 12:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:02:01.910473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9929c-84e9-4917-931e-de26e5816f22 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:02:01.912207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c0f9929c-84e9-4917-931e-de26e5816f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:02:01.923062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:02:01.971979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9929c-84e9-4917-931e-de26e5816f22 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:02:01.975495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc2a8-37ba-4aed-825a-7c0aefeaf0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:02:01.976145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc2a8-37ba-4aed-825a-7c0aefeaf0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:02:01.976431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc2a8-37ba-4aed-825a-7c0aefeaf0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:02:01.976710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc2a8-37ba-4aed-825a-7c0aefeaf0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:02:01.985486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdc2a8-37ba-4aed-825a-7c0aefeaf0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:02:01.986961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdc2a8-37ba-4aed-825a-7c0aefeaf0dd service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:02:01.996175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 785/1536] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:02:04.280297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c63af2-9e71-458c-82bf-7b8d2471e6e7 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:02:04.280835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c63af2-9e71-458c-82bf-7b8d2471e6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:02:04.280835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c63af2-9e71-458c-82bf-7b8d2471e6e7 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:02:04.282172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 752/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 12:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:02:04.287543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22069f-c737-47c9-aac3-225eb76f315d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:02:04.287976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22069f-c737-47c9-aac3-225eb76f315d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:02:04.288170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22069f-c737-47c9-aac3-225eb76f315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:02:04.306492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dc22069f-c737-47c9-aac3-225eb76f315d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:02:04.307767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dc22069f-c737-47c9-aac3-225eb76f315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:02:04.316115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 786/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 12:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:02:04.582135 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c63af2-9e71-458c-82bf-7b8d2471e6e7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:02:04.583884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-46c63af2-9e71-458c-82bf-7b8d2471e6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:02:04.592174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=71 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:02:04.598803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c63af2-9e71-458c-82bf-7b8d2471e6e7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:02:04.602347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684b76e-1356-4781-99f9-0f1771e58a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:02:04.603067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684b76e-1356-4781-99f9-0f1771e58a4e service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:02:04.603357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684b76e-1356-4781-99f9-0f1771e58a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:02:04.603640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684b76e-1356-4781-99f9-0f1771e58a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:02:04.613385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684b76e-1356-4781-99f9-0f1771e58a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:02:04.614863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684b76e-1356-4781-99f9-0f1771e58a4e service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:02:04.624160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 753/1539] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:02:14.301561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=26 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:02:14.620581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:35.245862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.246508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.246508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.279891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:35.281467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:35.293853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 787/1540] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:35.303184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.303774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.303774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.332819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:35.334039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:35.347650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 754/1541] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:35.353423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:35.368936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45364d31-f98e-4aed-82fa-2a1a326ff750 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:35.375023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:35.376300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.376815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.377314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.378289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:35.378834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:35.381083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:35.402980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 788/1542] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:35.411534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:35.425051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b9f91-a536-4375-9fcd-75ae7e881996 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:35.434594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:35.435315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.435605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.435890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.436617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:35.438590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:35.480653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 755/1543] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:35.485563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:35.501293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7a90-c116-4ae5-b8b8-e5c907dc0658 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:35.592718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:35.593422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.593743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.594029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.594756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:35.596868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:35.624986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 789/1544] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:35.629694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:35.641168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2d9b0-1471-4422-b2ea-d860fb417f29 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:35.651699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f5fe8-3c6c-445c-81cf-d3e6838fb3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:35.652391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f5fe8-3c6c-445c-81cf-d3e6838fb3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.652682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f5fe8-3c6c-445c-81cf-d3e6838fb3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.652967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f5fe8-3c6c-445c-81cf-d3e6838fb3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.653697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f5fe8-3c6c-445c-81cf-d3e6838fb3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:35.655356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f5fe8-3c6c-445c-81cf-d3e6838fb3b1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:35.670759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 756/1545] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:35.675461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:35.688228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fedc9-e28c-4fe9-852f-93f640f4cdb9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:35.697089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:35.697816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.698052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.698321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.699117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:35.700981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:35.711137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba7b9-dcdb-4cfd-bbb6-de839d6574e9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.711443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba7b9-dcdb-4cfd-bbb6-de839d6574e9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.711711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba7b9-dcdb-4cfd-bbb6-de839d6574e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.740744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 790/1546] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:35.745520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:35.755768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f34ba7b9-dcdb-4cfd-bbb6-de839d6574e9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:35.756999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f34ba7b9-dcdb-4cfd-bbb6-de839d6574e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:35.769392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c13fa-bcda-4294-9ba8-3fd9ed186c20 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:35.778147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 757/1547] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:35.779476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a94c9-1af9-45c7-a971-20a3851257b0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.779764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a94c9-1af9-45c7-a971-20a3851257b0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.780044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a94c9-1af9-45c7-a971-20a3851257b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.782980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:35.783896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.784181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.784462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] PATH_INFO: `/v3/projects/c557faf1b3444d80acf497e589b51e70/users/b0ba10e29e7341ce82d9b453a91f5fff/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.843911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c557faf1b3444d80acf497e589b51e70, user_id=b0ba10e29e7341ce82d9b453a91f5fff, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:35.844992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:03:35.844992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:03:35.844992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:03:35.846985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46681b72-0ccf-4786-b47d-f286e8daa792 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:35.850217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-661a94c9-1af9-45c7-a971-20a3851257b0 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:35.851454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-661a94c9-1af9-45c7-a971-20a3851257b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:35.869536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 758/1548] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:35.875547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6c8b-c8a0-4098-a854-fb3b9853318f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.875851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6c8b-c8a0-4098-a854-fb3b9853318f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.876132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6c8b-c8a0-4098-a854-fb3b9853318f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.883904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 791/1549] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:35 2022] PUT /identity/v3/projects/c557faf1b3444d80acf497e589b51e70/users/b0ba10e29e7341ce82d9b453a91f5fff/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:35.885336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4262f-25f5-4342-a64b-2be65c050877 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:35.885633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4262f-25f5-4342-a64b-2be65c050877 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:35.885918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4262f-25f5-4342-a64b-2be65c050877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:35.912031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-dcac6c8b-c8a0-4098-a854-fb3b9853318f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:35.913250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dcac6c8b-c8a0-4098-a854-fb3b9853318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:35.927781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5ae4262f-25f5-4342-a64b-2be65c050877 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:35.929402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5ae4262f-25f5-4342-a64b-2be65c050877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.009705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 759/1550] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.015613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a235a55-961e-492f-ab7e-ce2fee662ffb None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.016205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a235a55-961e-492f-ab7e-ce2fee662ffb None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.016707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a235a55-961e-492f-ab7e-ce2fee662ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.039775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 792/1551] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.041488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a858-0e47-4f09-9926-fb738212cc8b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.041774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a858-0e47-4f09-9926-fb738212cc8b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.042052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a858-0e47-4f09-9926-fb738212cc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.043730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-5a235a55-961e-492f-ab7e-ce2fee662ffb None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.044959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5a235a55-961e-492f-ab7e-ce2fee662ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.057255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 760/1552] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.058640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534df758-aecc-430f-ba85-08dc7028afd6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.058957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534df758-aecc-430f-ba85-08dc7028afd6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.059238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534df758-aecc-430f-ba85-08dc7028afd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.074890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-28c3a858-0e47-4f09-9926-fb738212cc8b None None] MFA Rules not processed for user `b0ba10e29e7341ce82d9b453a91f5fff`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.076109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-28c3a858-0e47-4f09-9926-fb738212cc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.078516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-534df758-aecc-430f-ba85-08dc7028afd6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.079705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-534df758-aecc-430f-ba85-08dc7028afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.087957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 761/1553] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.089460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.089747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.090024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.104811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:36.108809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.110108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.118876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 762/1554] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.119884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.129411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a8435-e766-4c7e-8455-cf267b445874 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.133942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.134589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.134918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.135207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.135928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.136239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.137284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:03:36.137284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:03:36.137284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:03:36.138179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733ceec-844a-4e1d-a918-3a2b076b4447 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.149998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 793/1555] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.150172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 763/1556] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.151288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6255a4-d349-48d5-88e0-eb0df01af43e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.151416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.151588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.151696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6255a4-d349-48d5-88e0-eb0df01af43e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.151861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6255a4-d349-48d5-88e0-eb0df01af43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.151975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.170536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.171773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.173155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6d6255a4-d349-48d5-88e0-eb0df01af43e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.174462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6d6255a4-d349-48d5-88e0-eb0df01af43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.179458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 794/1557] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.181749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.183196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 764/1558] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.184567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df91adc-213e-48f0-8b54-bb2dc4946b6a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.184862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df91adc-213e-48f0-8b54-bb2dc4946b6a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.185146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df91adc-213e-48f0-8b54-bb2dc4946b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.193145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef85f-488c-406c-a668-0e4e512be170 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.197858 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.198640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.199357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.199357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.200000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.200311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.201931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5988cd-d9dc-432e-9292-f7163cd929ec admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.204661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6df91adc-213e-48f0-8b54-bb2dc4946b6a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.205848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6df91adc-213e-48f0-8b54-bb2dc4946b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.220990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 765/1559] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.221175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 795/1560] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.221175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.221175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.221175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.221467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.221467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.221467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.235550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.236765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.238355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.239664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.245467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 796/1561] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.249154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.251353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 766/1562] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.255446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.258805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cd7f3-ad36-4e2e-9d70-90fc061cdd70 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.262558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4244e-6163-4c74-8d7d-78d77821e55b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.263383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.264027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.264312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.264588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.265264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.266278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:03:36.266278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:03:36.266278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:03:36.267503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.267886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f774ce4-dc54-44fc-b44e-ff4497232a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.268532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f774ce4-dc54-44fc-b44e-ff4497232a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.268821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f774ce4-dc54-44fc-b44e-ff4497232a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.269107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f774ce4-dc54-44fc-b44e-ff4497232a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.269842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f774ce4-dc54-44fc-b44e-ff4497232a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.271797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f774ce4-dc54-44fc-b44e-ff4497232a6e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.297595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 797/1563] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.298426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.302017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 767/1564] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.303377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.303977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.303977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.322330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.323573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.353209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:36.360887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946ebb-fc22-4f0f-922b-fe51692649be admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.386966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 768/1565] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.387890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.398249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540148b-fd12-4b36-be6c-d8e930e2707b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.403274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2a293-556f-46a1-942a-0a4ed6fcffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.403954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2a293-556f-46a1-942a-0a4ed6fcffe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.404238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2a293-556f-46a1-942a-0a4ed6fcffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.404514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2a293-556f-46a1-942a-0a4ed6fcffe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.405204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2a293-556f-46a1-942a-0a4ed6fcffe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.407395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2a293-556f-46a1-942a-0a4ed6fcffe5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.433855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.434725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.435041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.435322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.436046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.436380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.440119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 769/1566] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.440303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.440417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.440417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.440417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.451612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 798/1567] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.452514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.462171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefdb98-acc8-4c17-8eda-0ae6628a9f3e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.468155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:36.469880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:36.477390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:36.526223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.526926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.527218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.527502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.528223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.528536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.530275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.532325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 770/1568] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:36.533171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.543416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f8bb5-4d8d-4f51-adf8-4963338f6df5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.545705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 799/1569] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.546345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.555232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.555727 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3d19-de30-4fbe-9c3a-6b231eff9d70 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.555936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.556224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.556614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.557342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.557653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.559315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.560363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.561003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.561293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.561576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.562254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.564188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.571072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 771/1570] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.572191 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.583396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992f5c-cdf2-4bd9-a33d-0d417f8382f9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.589035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.589731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.589856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 800/1571] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.590015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.590293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.590798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.591007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.592054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:03:36.592054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:03:36.592054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:03:36.593265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.600274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9273ae-3ffc-4efd-a400-83b4bd48f5c7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.605057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.605744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.605986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.606273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.607032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.607348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.608964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.617209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 772/1572] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.617209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.630091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 801/1573] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.630091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.630362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea12fd-358e-445e-bb06-cc40d54f2e63 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.630541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.631243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.631532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.631814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.632515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.633756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17632e3e-5205-4fd0-9d66-442e31bf4ce5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.634761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.638522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.639181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.639478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.639757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.640437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.642328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.654604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 773/1574] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.655512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.665624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab5e3-c2b8-4bc1-97cb-bf0811a6d528 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.669213 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 802/1575] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.669884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.671577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.672377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.672737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.673086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.673869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:36.674247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.675952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.688197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 774/1576] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:36.689154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.699350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c194df2-53a5-4065-bfd4-a16f9c8d8e06 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.733780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472de05-d2f1-4951-8504-5fd4abadabd1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.738397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.739096 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.739384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.739668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.740377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.742295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.744775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:36.755170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.755828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.756136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.756416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.757313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.759462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.781737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 803/1577] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.781737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.794137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 775/1578] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.794137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.833164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:36.853002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d402600-4b82-4d06-8183-754111d56e1d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.857724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.858402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.858719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.859026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.859751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.861380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.861659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52693-815f-4144-9186-fe92387a4647 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.866247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.866943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.867280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.867511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.868289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.870030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.871683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 804/1579] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:36.872618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.880604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 776/1580] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:36.881421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.884693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5962-bb7a-45b7-ae5e-1fc3bca44d98 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.889279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.889928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.890218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.890504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.891308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.891542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10132abb-8f92-49fe-a3a8-30c45a2005e4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.893233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.896064 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.896698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.896987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.897279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.897983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.900287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.910586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 777/1581] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:36.911624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.921383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 805/1582] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:36.921791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46059f-000e-4648-8b10-6e26c1a696ad admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.922235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:36.982518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.983238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.983528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.983816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.984521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.986466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:36.987330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f5d9-6d6c-4047-a356-6fd751c7dcc0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:36.993289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:36.994253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:36.994921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:36.995265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:36.996203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:36.998594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.014788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 778/1583] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:37.014788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.025235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc63544-68b7-4bfb-b55c-ad2a73423e80 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.031840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.033130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.033662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.033878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 806/1584] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:36 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:37.034234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.034576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.035601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.038347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.061801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 779/1585] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/users => generated 297 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:37.063206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.074119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09322585-7511-4557-b1a7-3d6cbf6711bd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.103384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd8ffc-797a-462a-9fcf-3e8937194e7c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.108721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.109678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.110058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.110354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.111366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.112662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:37.114504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.137593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 807/1586] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:37.141564 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.148137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.148828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.149121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.149435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.150538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.151917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.152481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead2a0-9f21-48db-8e2f-59862cbce4a1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.156804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.157681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.157968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.158254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] PATH_INFO: `/v3/projects/281a0fc992a7467595db80fb51b443de/users/893e8c08d5964233b6b9306e9c83c099/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.165912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 780/1587] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.165912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.173120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41a484-ec8c-4f6d-a159-b2a4e75c6b89 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.173790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281a0fc992a7467595db80fb51b443de, user_id=893e8c08d5964233b6b9306e9c83c099, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.176876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.177950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.178630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.179015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.179299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.180018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.182533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.191185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 808/1588] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/281a0fc992a7467595db80fb51b443de/users/893e8c08d5964233b6b9306e9c83c099/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.192223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.199431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 781/1589] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.199431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.205423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:37.208835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87031a2-335a-41e6-88f1-250d771629d0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.259349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ff73f-1079-49bc-a1cc-393c061104e4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.264226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.264948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.265236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.265541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.266236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.267597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.268246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.268351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.268540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.268822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.269544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.271205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.282032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 782/1590] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.282832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.295014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 809/1591] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:37.295014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.295314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44957be1-c1d6-4e67-9ee1-4b9f5e3b3c9b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.297804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.298456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.298743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.299056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.299739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.301656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.303979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b41303-0161-4d63-aae3-b0fe12e43b3e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.308576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.309348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.309706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.309996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.310713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.312453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.322316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 783/1592] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:37.326319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.326415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 810/1593] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.326415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.335206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96bc5a-3dc8-43d9-8bf0-84e2bc42956e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.335375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020da9f-f121-4c83-86bf-10298eeab01d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.340878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.341343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.341532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.341824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.342110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.342286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.342554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.342826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.342968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] PATH_INFO: `/v3/projects/ffaf09bc77914692bb965b55af538141/users/6c904a413f6a4ba49686debac817c70d/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.344455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.355955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 784/1594] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.356915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.361240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffaf09bc77914692bb965b55af538141, user_id=6c904a413f6a4ba49686debac817c70d, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.364035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.366631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266538e8-df92-49c8-9cc1-3cb52f06ac8b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.372780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.373618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.373908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.374188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.374934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.376530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.377825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 811/1595] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/ffaf09bc77914692bb965b55af538141/users/6c904a413f6a4ba49686debac817c70d/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.378774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.387914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 785/1596] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.388834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.398175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87321d22-0877-48a3-ad52-4c7ee6cc9d6d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.444786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd24404-4df0-48a3-a197-02310cd70df0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.450748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.451703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.451993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.452273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] PATH_INFO: `/v3/projects/b7d38f1b5a8a43ea8677f3ce053347da/users/6aebb057a324492699141a878793c097/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.457142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.457891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.458256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.458610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.459442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.461159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.468683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d38f1b5a8a43ea8677f3ce053347da, user_id=6aebb057a324492699141a878793c097, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.471445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.471643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 786/1597] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.472458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.483355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80a47-5b3e-42a8-bfb3-9647f825623b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.485874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 812/1598] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/b7d38f1b5a8a43ea8677f3ce053347da/users/6aebb057a324492699141a878793c097/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.487679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.491531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:37.537973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:37.546283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.546983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.547276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.547551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.548271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.549910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.554402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f46a-b16f-425e-bec6-6bd149418dde admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.559805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.560470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.560966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.561086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 787/1599] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.561175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.561562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.561745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.563394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.570794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fed2a-e1f8-4bda-9bc3-a035ec7ab0df admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.576906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 813/1600] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.576906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.578776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c797f7-8277-4bbf-9d24-e5088165d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.579446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c797f7-8277-4bbf-9d24-e5088165d7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.579730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c797f7-8277-4bbf-9d24-e5088165d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.580005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c797f7-8277-4bbf-9d24-e5088165d7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.580716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c797f7-8277-4bbf-9d24-e5088165d7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.582293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c797f7-8277-4bbf-9d24-e5088165d7c5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.584179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eda28-8590-46dc-9606-d4ffa968eada admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.590264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.591182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.591529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.591810 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.592577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.593628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 788/1601] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.594181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.594897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.595216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.595469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.605550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 814/1602] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.606364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.614993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:37.616211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0e340-2a06-427b-96a8-4a5f987f3289 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.616337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:37.620732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d6b3b-7d6f-4d36-8977-635be867d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.621584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d6b3b-7d6f-4d36-8977-635be867d1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.621868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d6b3b-7d6f-4d36-8977-635be867d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.622138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d6b3b-7d6f-4d36-8977-635be867d1dd admin admin] PATH_INFO: `/v3/projects/281a0fc992a7467595db80fb51b443de/users/893e8c08d5964233b6b9306e9c83c099/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.624087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d6b3b-7d6f-4d36-8977-635be867d1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=281a0fc992a7467595db80fb51b443de, user_id=893e8c08d5964233b6b9306e9c83c099, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.624277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 789/1603] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:37.627269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d6b3b-7d6f-4d36-8977-635be867d1dd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.627679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.637857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1a1d-29a8-4f3f-b233-36c1f95c2efc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.639582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 815/1604] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/281a0fc992a7467595db80fb51b443de/users/893e8c08d5964233b6b9306e9c83c099/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.640886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.641172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.641451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.660080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:37.661297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:37.701072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.702120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.702431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.702713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] PATH_INFO: `/v3/projects/30f4ac789e8e41468a98175ef8d60d18/users/3ba79450153f4399bfe9d3900ed12523/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.717424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f4ac789e8e41468a98175ef8d60d18, user_id=3ba79450153f4399bfe9d3900ed12523, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.720148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.722331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 816/1605] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:37.723196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.734156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8b4a4-b56c-418e-a042-79d80b8a9870 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.735380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 790/1606] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/30f4ac789e8e41468a98175ef8d60d18/users/3ba79450153f4399bfe9d3900ed12523/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.736082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.795512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.796465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.796757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.797039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.797792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.799530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.800448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ebd84-b9b0-4ae0-beae-ac680d97713d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.805371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.806188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.806475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.806786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.807500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.809092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.813271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 817/1607] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.813271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.820410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df52293-6c3c-4baf-9050-ea8f09120384 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.826129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 791/1608] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:37.826129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.826385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.826523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.826795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.827109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] PATH_INFO: `/v3/projects/cf5de9215c3243e1b2fabc859f102b2a/users/473532c21cad4c3492c8425855993b0f/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.834535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ee63-7413-4136-86ea-5f7093f73fe5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.840446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a16344-0e08-4868-936e-f18691ca4e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.841069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a16344-0e08-4868-936e-f18691ca4e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.841359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a16344-0e08-4868-936e-f18691ca4e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.841650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a16344-0e08-4868-936e-f18691ca4e81 admin admin] PATH_INFO: `/v3/projects/086d355c6b804aefb0b53d2f7848ed0e/users/ac1534ff6fcd4283b36e2c6d4e019085/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.843319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5de9215c3243e1b2fabc859f102b2a, user_id=473532c21cad4c3492c8425855993b0f, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.844793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:37.846059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.858364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a16344-0e08-4868-936e-f18691ca4e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=086d355c6b804aefb0b53d2f7848ed0e, user_id=ac1534ff6fcd4283b36e2c6d4e019085, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:37.859509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 818/1609] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/cf5de9215c3243e1b2fabc859f102b2a/users/473532c21cad4c3492c8425855993b0f/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.860462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.861148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a16344-0e08-4868-936e-f18691ca4e81 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:37.874750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 792/1610] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:37 2022] PUT /identity/v3/projects/086d355c6b804aefb0b53d2f7848ed0e/users/ac1534ff6fcd4283b36e2c6d4e019085/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:37.879662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.879960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.880252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.895192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:37.907544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] MFA Rules not processed for user `893e8c08d5964233b6b9306e9c83c099`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:37.908761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:37.919180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdec1b-e8b2-490e-8f1e-b7b542f0b305 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.971994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fecf26-e12f-4295-b738-5d366e34898a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.972650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fecf26-e12f-4295-b738-5d366e34898a service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.972939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fecf26-e12f-4295-b738-5d366e34898a service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.973224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fecf26-e12f-4295-b738-5d366e34898a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:37.980607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 793/1611] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:37.981649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:37.991415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b3043-3148-4ce8-8e58-1e65eac44fc0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:37.995335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:37.995980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:37.996266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:37.996548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.040502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fecf26-e12f-4295-b738-5d366e34898a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.041973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fecf26-e12f-4295-b738-5d366e34898a service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.051573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 819/1612] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.059348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.059911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.060206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.061715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.063245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.074176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 794/1613] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.075017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.080100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:38.081377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:38.084737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c8ff5-7278-4e2b-a44b-fe09ac551679 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.090531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 820/1614] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:38.090693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0c556-13ad-45c2-b28f-65c688c20185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.091541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c556-13ad-45c2-b28f-65c688c20185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.091809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.091929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c556-13ad-45c2-b28f-65c688c20185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.092097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c556-13ad-45c2-b28f-65c688c20185 admin admin] PATH_INFO: `/v3/projects/e4e76a3125df4436abf5f788e35d492f/users/c8bda0c079054e198ffeca4c0ac18bf2/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.104272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd9ff9-9a67-4bea-8886-97a8fd0e05be None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.107971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0c556-13ad-45c2-b28f-65c688c20185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e76a3125df4436abf5f788e35d492f, user_id=c8bda0c079054e198ffeca4c0ac18bf2, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.109617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d60ce-9d53-49f2-8488-6180dca452ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.110518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d60ce-9d53-49f2-8488-6180dca452ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.110690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0c556-13ad-45c2-b28f-65c688c20185 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.110823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d60ce-9d53-49f2-8488-6180dca452ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.111116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d60ce-9d53-49f2-8488-6180dca452ba admin admin] PATH_INFO: `/v3/projects/da3be8c138a045d4a39365a67f6b0c91/users/f104e2702bd945669ebb93f8f5f40421/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.125805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 795/1615] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:38 2022] PUT /identity/v3/projects/e4e76a3125df4436abf5f788e35d492f/users/c8bda0c079054e198ffeca4c0ac18bf2/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:38.129463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d99261-4528-4277-90ea-f7cadad417de None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.129463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d99261-4528-4277-90ea-f7cadad417de None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.129463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d99261-4528-4277-90ea-f7cadad417de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.129996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d60ce-9d53-49f2-8488-6180dca452ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3be8c138a045d4a39365a67f6b0c91, user_id=f104e2702bd945669ebb93f8f5f40421, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.132925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d60ce-9d53-49f2-8488-6180dca452ba admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.146415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-45d99261-4528-4277-90ea-f7cadad417de None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:38.147593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-45d99261-4528-4277-90ea-f7cadad417de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:38.148901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 821/1616] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:38 2022] PUT /identity/v3/projects/da3be8c138a045d4a39365a67f6b0c91/users/f104e2702bd945669ebb93f8f5f40421/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:38.150703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.151061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.151357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.170250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:38.171705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:38.187709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:38.223579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 796/1617] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:38.224660 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d99261-4528-4277-90ea-f7cadad417de None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.270614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 822/1618] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:38.271544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.273017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:38.290669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:38.291701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d99261-4528-4277-90ea-f7cadad417de None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.298810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f263-2c0e-4884-bfc8-8ee0eafa5779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.299663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f263-2c0e-4884-bfc8-8ee0eafa5779 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.300058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f263-2c0e-4884-bfc8-8ee0eafa5779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.300578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f263-2c0e-4884-bfc8-8ee0eafa5779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.356310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739934fe-b9cc-417d-9e3f-8e7ebab48801 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.361086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d9ec6-948a-49d4-9f70-85af998c4868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.361817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d9ec6-948a-49d4-9f70-85af998c4868 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.362188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d9ec6-948a-49d4-9f70-85af998c4868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.362542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d9ec6-948a-49d4-9f70-85af998c4868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.372991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d9ec6-948a-49d4-9f70-85af998c4868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.374782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d9ec6-948a-49d4-9f70-85af998c4868 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.378876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f263-2c0e-4884-bfc8-8ee0eafa5779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.380370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f263-2c0e-4884-bfc8-8ee0eafa5779 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.388228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 823/1619] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.392652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 797/1620] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.393246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.393987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.394350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.394514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.394795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.395037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.421744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:38.423290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:38.427303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] MFA Rules not processed for user `f104e2702bd945669ebb93f8f5f40421`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:38.428821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:38.433474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 824/1621] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:38.436667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.450634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67095b5c-eae0-49c3-bd24-2eab273c5fe3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.456303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.457324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.457771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.458115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.473553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.475090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.487803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 825/1622] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.488778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.500111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68a8af-5177-4ff9-b8e0-b56aa55e0271 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.506226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.507084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.507441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.507845 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.550243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.560600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.580139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 798/1623] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:38.585097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.591236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 826/1624] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.593827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.596789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3749ba-a6ac-4d1f-92cb-54cd8b3a2516 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.603200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.608425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.608925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.609344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.626531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f834b-8363-4c16-9e0c-9df0bc05806c service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.644080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.646032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.647387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.647907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.648199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.652293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.654163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.660932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.686720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 799/1625] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:38.727905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.743881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 827/1626] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:38.749738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c417b6-01dd-4911-b02e-30d54243686a service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.755810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:38.903819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.905210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.905610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.905905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.907025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.913328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:38.957677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 800/1627] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:38 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:38.958720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:38.975039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ae981-84e7-4baa-94e8-b401f7d7a0bb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:38.984970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:38.985955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:38.986469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:38.987016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:38.988131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:38.990670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.017580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 828/1628] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:03:39.018673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.041219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e98b5-10b1-4e3a-aab9-0d45199e7a49 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.053424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.056229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.056682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.058246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] PATH_INFO: `/v3/projects/6aa5b6ab113343988fafcd37ad5ea92e/users/723597a7d38e4d1b9af0635adc90eb72/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.083045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa5b6ab113343988fafcd37ad5ea92e, user_id=723597a7d38e4d1b9af0635adc90eb72, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.086671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.108944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 801/1629] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:39 2022] PUT /identity/v3/projects/6aa5b6ab113343988fafcd37ad5ea92e/users/723597a7d38e4d1b9af0635adc90eb72/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:39.113670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.130239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1429b9-f410-4eef-9879-684f0399c799 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.133233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.223153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.224356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.224628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.224989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.225870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.228397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.255018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 829/1630] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:39.257290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.284059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2804227-4210-4dff-b946-16cc7bcf539e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.289332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.290616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.290957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.291393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] PATH_INFO: `/v3/users/b0ba10e29e7341ce82d9b453a91f5fff` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.292897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ba10e29e7341ce82d9b453a91f5fff)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.294907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.295373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b4bc0-325e-4592-ab86-868ad6e853b1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.345376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688173-e7ce-4459-8ee3-58e5d6a56b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.347659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688173-e7ce-4459-8ee3-58e5d6a56b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.350350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688173-e7ce-4459-8ee3-58e5d6a56b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.351789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688173-e7ce-4459-8ee3-58e5d6a56b4c admin admin] PATH_INFO: `/v3/projects/6aa5b6ab113343988fafcd37ad5ea92e/users/723597a7d38e4d1b9af0635adc90eb72/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.362282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688173-e7ce-4459-8ee3-58e5d6a56b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa5b6ab113343988fafcd37ad5ea92e, user_id=723597a7d38e4d1b9af0635adc90eb72, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.365839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688173-e7ce-4459-8ee3-58e5d6a56b4c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.379187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ba10e29e7341ce82d9b453a91f5fff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:39.380446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ba10e29e7341ce82d9b453a91f5fff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:39.381419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ba10e29e7341ce82d9b453a91f5fff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:39.404545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ba10e29e7341ce82d9b453a91f5fff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:39.404742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 830/1631] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:39 2022] PUT /identity/v3/projects/6aa5b6ab113343988fafcd37ad5ea92e/users/723597a7d38e4d1b9af0635adc90eb72/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:03:39.405721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.406038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.406397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.417712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4cdfed0b-6e17-4c75-90a5-baeea27f3a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ba10e29e7341ce82d9b453a91f5fff'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:39.419744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:39.465364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] MFA Rules not processed for user `723597a7d38e4d1b9af0635adc90eb72`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:39.468214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:39.469477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 802/1632] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:39 2022] DELETE /identity/v3/users/b0ba10e29e7341ce82d9b453a91f5fff => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:39.486749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.488769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.489275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.505058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:39.533936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:39.535697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:39.646220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 831/1633] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:03:39.663396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 803/1634] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:39.663615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.666403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:39.708314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.712920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:39.781301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30859b8-17cd-4b93-9092-2f2481ccee66 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.792734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.797028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.798771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.799163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.846783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a4f8f-fd1e-42b8-bbc3-4b30c81d3615 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.849962 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.851918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.865159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efd5f1-7684-4bf0-a7d4-c29692922dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.867606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efd5f1-7684-4bf0-a7d4-c29692922dbe service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.867913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efd5f1-7684-4bf0-a7d4-c29692922dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.868252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efd5f1-7684-4bf0-a7d4-c29692922dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.885275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efd5f1-7684-4bf0-a7d4-c29692922dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.887709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efd5f1-7684-4bf0-a7d4-c29692922dbe service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:39.900444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 832/1635] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:39.904781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 804/1636] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:39.934232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:39.936759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:39.972407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a70c0-2d04-4fb9-8e14-beec7958346b service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:39.980656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:39.981724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:39.982139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:39.982618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] PATH_INFO: `/v3/users/f104e2702bd945669ebb93f8f5f40421` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:39.984337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f104e2702bd945669ebb93f8f5f40421)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:39.986995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: WARNING py.warnings [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:03:39.986995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: warnings.warn(msg) Jun 06 12:03:39.986995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: Jun 06 12:03:39.988452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:40.125927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f104e2702bd945669ebb93f8f5f40421'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:40.146197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f104e2702bd945669ebb93f8f5f40421'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:40.172176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f104e2702bd945669ebb93f8f5f40421'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:40.173329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f104e2702bd945669ebb93f8f5f40421'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:40.184162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e83fc01d-7dbb-417d-b40c-48763c3d5e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f104e2702bd945669ebb93f8f5f40421'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:40.205667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918a572-d0ca-4005-a42e-4dcf74ad7ed2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.206360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918a572-d0ca-4005-a42e-4dcf74ad7ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.206804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918a572-d0ca-4005-a42e-4dcf74ad7ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.242219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 833/1637] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:39 2022] DELETE /identity/v3/users/f104e2702bd945669ebb93f8f5f40421 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:40.249904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a332-6d56-4702-868e-41ed76d3f3f6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.250505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a332-6d56-4702-868e-41ed76d3f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.250985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a332-6d56-4702-868e-41ed76d3f3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.258330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a918a572-d0ca-4005-a42e-4dcf74ad7ed2 None None] MFA Rules not processed for user `3ba79450153f4399bfe9d3900ed12523`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:40.260050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a918a572-d0ca-4005-a42e-4dcf74ad7ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:40.298941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8383a332-6d56-4702-868e-41ed76d3f3f6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:40.302759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8383a332-6d56-4702-868e-41ed76d3f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:40.356981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:40.392551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 805/1638] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:03:40.398329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d832b8-64b6-429c-a048-28e0cc8d9784 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.398496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d832b8-64b6-429c-a048-28e0cc8d9784 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.398889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d832b8-64b6-429c-a048-28e0cc8d9784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.440681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-02d832b8-64b6-429c-a048-28e0cc8d9784 None None] MFA Rules not processed for user `3ba79450153f4399bfe9d3900ed12523`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:40.443587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-02d832b8-64b6-429c-a048-28e0cc8d9784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:40.459931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 834/1639] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:40.464062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 806/1640] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:40.496638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09fd6-51af-4bb3-a867-a0e1a74c627a None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.497339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09fd6-51af-4bb3-a867-a0e1a74c627a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.497834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09fd6-51af-4bb3-a867-a0e1a74c627a None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.500079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 835/1641] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 12:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:40.517488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.518352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.518352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.545377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a09fd6-51af-4bb3-a867-a0e1a74c627a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:40.547431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-87a09fd6-51af-4bb3-a867-a0e1a74c627a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:40.562461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:40.567078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:40.603390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:40.679228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a09fd6-51af-4bb3-a867-a0e1a74c627a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:40.691856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe3294-9d1a-4229-aa1b-7faa2055ac77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:40.693643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe3294-9d1a-4229-aa1b-7faa2055ac77 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.695320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe3294-9d1a-4229-aa1b-7faa2055ac77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.696054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe3294-9d1a-4229-aa1b-7faa2055ac77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.727230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe3294-9d1a-4229-aa1b-7faa2055ac77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:40.729825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe3294-9d1a-4229-aa1b-7faa2055ac77 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:40.749173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 807/1642] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:40.751060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:40.768329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 836/1643] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:40.775063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484652e-4311-4d6e-80c1-5acf7c8fa3b4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:40.782145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8d2dc-9afe-405c-bd7a-b1f6ae7734c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:40.782725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8d2dc-9afe-405c-bd7a-b1f6ae7734c1 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:40.783133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8d2dc-9afe-405c-bd7a-b1f6ae7734c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:40.783514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8d2dc-9afe-405c-bd7a-b1f6ae7734c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:40.795304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8d2dc-9afe-405c-bd7a-b1f6ae7734c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:40.798687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8d2dc-9afe-405c-bd7a-b1f6ae7734c1 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:40.822801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 808/1644] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:41.143274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1c23b-0d14-4192-ad3b-3c8b0bfa013d None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:41.143712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1c23b-0d14-4192-ad3b-3c8b0bfa013d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:41.144108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1c23b-0d14-4192-ad3b-3c8b0bfa013d None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:41.146759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 837/1645] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 12:03:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:41.154470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:41.154951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:41.155348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:41.179152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:41.180862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:41.192610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 809/1646] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:41.209639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1c23b-0d14-4192-ad3b-3c8b0bfa013d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:41.213690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-34a1c23b-0d14-4192-ad3b-3c8b0bfa013d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:41.300105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1c23b-0d14-4192-ad3b-3c8b0bfa013d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:41.308501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:41.309924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:41.311350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:41.312454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:41.332917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:41.335079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:41.349137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:41.350673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:41.356361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 838/1647] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:41.382558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c82bf6-32a6-4824-b355-bb812bdee289 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:41.415512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:41.416608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:41.417004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:41.418965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] PATH_INFO: `/v3/projects/c557faf1b3444d80acf497e589b51e70` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:41.421544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c557faf1b3444d80acf497e589b51e70)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:41.425797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: WARNING py.warnings [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 12:03:41.425797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: warnings.warn(msg) Jun 06 12:03:41.425797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: Jun 06 12:03:41.427394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:41.505432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c557faf1b3444d80acf497e589b51e70', 'request_id': 'req-ee0d9db6-ea7c-4609-8160-17593743d9e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.513206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c557faf1b3444d80acf497e589b51e70', 'request_id': 'req-ee0d9db6-ea7c-4609-8160-17593743d9e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.528655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c557faf1b3444d80acf497e589b51e70', 'request_id': 'req-ee0d9db6-ea7c-4609-8160-17593743d9e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.548016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] The token cache is being invalidate because project c557faf1b3444d80acf497e589b51e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:03:41.548485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.550244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ee0d9db6-ea7c-4609-8160-17593743d9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.556572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 810/1648] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:41 2022] DELETE /identity/v3/projects/c557faf1b3444d80acf497e589b51e70 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:41.557902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:41.575182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52fc17-c32a-4ed2-a868-148535242336 service glance] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:41.717689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:41.719685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:41.720141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:41.721763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] PATH_INFO: `/v3/users/893e8c08d5964233b6b9306e9c83c099` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:41.724153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893e8c08d5964233b6b9306e9c83c099)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:41.727168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:41.767231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:41.840001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893e8c08d5964233b6b9306e9c83c099'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.841356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893e8c08d5964233b6b9306e9c83c099'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.851719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893e8c08d5964233b6b9306e9c83c099'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.855033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893e8c08d5964233b6b9306e9c83c099'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.875573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893e8c08d5964233b6b9306e9c83c099'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:41.923014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 839/1649] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:41 2022] DELETE /identity/v3/users/893e8c08d5964233b6b9306e9c83c099 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:42.089306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d3be-f9e9-4f03-ac9e-6287b5513204 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:42.089591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d3be-f9e9-4f03-ac9e-6287b5513204 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:42.089876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d3be-f9e9-4f03-ac9e-6287b5513204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:42.149090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d4d6d3be-f9e9-4f03-ac9e-6287b5513204 None None] MFA Rules not processed for user `ac1534ff6fcd4283b36e2c6d4e019085`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:42.157008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d4d6d3be-f9e9-4f03-ac9e-6287b5513204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:42.243391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:42.246977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:42.335754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 811/1650] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:03:42.339783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98a28b-08e6-4e46-b4c2-9ce89a25a571 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:42.340706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98a28b-08e6-4e46-b4c2-9ce89a25a571 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:42.341107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98a28b-08e6-4e46-b4c2-9ce89a25a571 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:42.343380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 812/1651] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 12:03:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:42.354783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98a28b-08e6-4e46-b4c2-9ce89a25a571 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:42.358230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bb98a28b-08e6-4e46-b4c2-9ce89a25a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:42.372820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:42.429493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afdb7e-c877-4c57-86f1-7a52de047c51 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:42.444906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:42.445529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:42.445823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:42.446654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] PATH_INFO: `/v3/projects/da3be8c138a045d4a39365a67f6b0c91` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:42.452317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3be8c138a045d4a39365a67f6b0c91)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:42.454383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:42.488363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98a28b-08e6-4e46-b4c2-9ce89a25a571 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:42.497052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b1505-dc85-4f7e-8b5c-41e71c8118c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:42.497696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1505-dc85-4f7e-8b5c-41e71c8118c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:42.497986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1505-dc85-4f7e-8b5c-41e71c8118c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:42.498609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1505-dc85-4f7e-8b5c-41e71c8118c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:42.499562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b1505-dc85-4f7e-8b5c-41e71c8118c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:42.504964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b1505-dc85-4f7e-8b5c-41e71c8118c2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:42.561333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3be8c138a045d4a39365a67f6b0c91', 'request_id': 'req-542ea1d7-089d-4d03-b597-611121561ac0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:42.568784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 813/1652] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:42 2022] POST /identity/v3/projects => generated 403 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:42.570471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3be8c138a045d4a39365a67f6b0c91', 'request_id': 'req-542ea1d7-089d-4d03-b597-611121561ac0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:42.572375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:42.572723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:42.573013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:42.583277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3be8c138a045d4a39365a67f6b0c91', 'request_id': 'req-542ea1d7-089d-4d03-b597-611121561ac0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:42.604323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] The token cache is being invalidate because project da3be8c138a045d4a39365a67f6b0c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:03:42.604894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:42.606744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:42.609126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 840/1653] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:42 2022] DELETE /identity/v3/projects/da3be8c138a045d4a39365a67f6b0c91 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:03:42.611373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:42.612999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:42.623592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:42.626170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:42.704737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:42.773783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 814/1654] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:03:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:42.800567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:42.803749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:42.831815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ea1d7-089d-4d03-b597-611121561ac0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:42.990045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:42.990956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:42.992405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:42.994325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:42.994722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:42.998020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8553de-6e34-45f2-9cf3-ca668f18021c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:42.999474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.017559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:43.018807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:43.019352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:43.019879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] PATH_INFO: `/v3/users/723597a7d38e4d1b9af0635adc90eb72` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:43.022619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] RBAC: Authorizing `identity:delete_user(user_id=723597a7d38e4d1b9af0635adc90eb72)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:43.028486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.057939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 841/1655] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:42 2022] POST /identity/v3/users => generated 308 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:43.060084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:43.064088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:43.078735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f517878-801e-4e51-93a2-db222d23eec5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:43.085078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:43.086459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:43.086647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:43.088305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:43.106561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:43.109086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.115599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:43.121559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 842/1656] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:43.122618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:43.133920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723597a7d38e4d1b9af0635adc90eb72'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:43.134618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723597a7d38e4d1b9af0635adc90eb72'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:43.136905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a72f1-5e6a-4a7b-bf41-d4a4dd45b728 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:43.137409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723597a7d38e4d1b9af0635adc90eb72'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:43.143313 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:43.144574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:43.144574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:43.144814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:43.145779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:43.148353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.153468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723597a7d38e4d1b9af0635adc90eb72'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:43.163683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 843/1657] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:43.169147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:43.176838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:43.181430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723597a7d38e4d1b9af0635adc90eb72'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:43.201523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6227d5-b61e-4cc0-b49f-52c6b59bc227 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:43.208782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:43.209703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:43.210048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:43.210493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] PATH_INFO: `/v3/projects/c2dd1404952a426cab6d3df84da39983/users/7372171c6dd44f398ca219ad5502a240/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:43.224440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 815/1658] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:42 2022] DELETE /identity/v3/users/723597a7d38e4d1b9af0635adc90eb72 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:43.233660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2dd1404952a426cab6d3df84da39983, user_id=7372171c6dd44f398ca219ad5502a240, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:43.241241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.267259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 844/1659] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:43 2022] PUT /identity/v3/projects/c2dd1404952a426cab6d3df84da39983/users/7372171c6dd44f398ca219ad5502a240/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:43.271402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:43.391466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e7555-4a6e-45c5-9def-e82f9fc4400f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:43.502354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:43.504520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:43.507473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:43.508015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:43.509617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:43.512302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.546088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 816/1660] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:43.551006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:43.571325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925d0c6-d101-45dc-989c-c09eeffefc4c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:43.576887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:43.578643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:43.579117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:43.579520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] PATH_INFO: `/v3/projects/c2dd1404952a426cab6d3df84da39983/users/7372171c6dd44f398ca219ad5502a240/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:43.582431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2dd1404952a426cab6d3df84da39983, user_id=7372171c6dd44f398ca219ad5502a240, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:43.586242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:43.613589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 845/1661] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:43 2022] PUT /identity/v3/projects/c2dd1404952a426cab6d3df84da39983/users/7372171c6dd44f398ca219ad5502a240/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:03:43.806344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:43.811319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:43.871006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6c3d-2d8e-45d1-b73e-71b29a418911 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:44.014108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:44.017099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:44.105054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:44.106647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.107014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.107431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] PATH_INFO: `/v3/projects/281a0fc992a7467595db80fb51b443de` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.108948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=281a0fc992a7467595db80fb51b443de)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:44.111822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:44.114111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:44.154606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:44.160192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4158f2-2b7d-4d2d-9994-29a9ed879de8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:44.176915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:44.177848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.178139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.178630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] PATH_INFO: `/v3/projects/6aa5b6ab113343988fafcd37ad5ea92e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.180002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa5b6ab113343988fafcd37ad5ea92e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:44.182801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:44.255284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281a0fc992a7467595db80fb51b443de', 'request_id': 'req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.278855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281a0fc992a7467595db80fb51b443de', 'request_id': 'req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.283784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa5b6ab113343988fafcd37ad5ea92e', 'request_id': 'req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.295963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281a0fc992a7467595db80fb51b443de', 'request_id': 'req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.301823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa5b6ab113343988fafcd37ad5ea92e', 'request_id': 'req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.329111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa5b6ab113343988fafcd37ad5ea92e', 'request_id': 'req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.341081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] The token cache is being invalidate because project 281a0fc992a7467595db80fb51b443de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:03:44.341404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.348024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b5a64c86-4ca3-4bb2-92f7-25c6509c5598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.352554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] The token cache is being invalidate because project 6aa5b6ab113343988fafcd37ad5ea92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:03:44.352851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.357498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 817/1662] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:43 2022] DELETE /identity/v3/projects/281a0fc992a7467595db80fb51b443de => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:44.357674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0988d4f5-f4f9-42f9-b6f1-eb53706a9ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:44.361653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21863927-8542-4e21-ac67-45dd0894b838 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.362004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21863927-8542-4e21-ac67-45dd0894b838 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.365702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 846/1663] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:44 2022] DELETE /identity/v3/projects/6aa5b6ab113343988fafcd37ad5ea92e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:44.365964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21863927-8542-4e21-ac67-45dd0894b838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.369658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7a204-b65a-4913-9240-e6b98b258360 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.369952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7a204-b65a-4913-9240-e6b98b258360 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.370343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7a204-b65a-4913-9240-e6b98b258360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.426080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-21863927-8542-4e21-ac67-45dd0894b838 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:44.427325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21863927-8542-4e21-ac67-45dd0894b838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:44.445908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-61b7a204-b65a-4913-9240-e6b98b258360 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:44.447867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-61b7a204-b65a-4913-9240-e6b98b258360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:44.682269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 818/1664] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:44.682269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf326e-4e79-4589-ae2c-5a9da9a96006 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.682269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf326e-4e79-4589-ae2c-5a9da9a96006 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.682269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf326e-4e79-4589-ae2c-5a9da9a96006 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.685432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 847/1665] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:44.685432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.685432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.685432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.686265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 819/1666] 10.222.0.41 () {54 vars in 960 bytes} [Mon Jun 6 12:03:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:44.691277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4437-7512-4c8f-8c4f-3d725771afd7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.691570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4437-7512-4c8f-8c4f-3d725771afd7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.691836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4437-7512-4c8f-8c4f-3d725771afd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.747374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:44.748629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:44.762279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-229b4437-7512-4c8f-8c4f-3d725771afd7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:44.770773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-229b4437-7512-4c8f-8c4f-3d725771afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:44.792387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 820/1667] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:44.794661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.797986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.799505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.845756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:44.848283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:44.869229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 821/1668] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:44.880447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:44.913335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 848/1669] 10.222.0.41 () {58 vars in 1057 bytes} [Mon Jun 6 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:03:44.914018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:44.937084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bc05e-ecfa-4251-bfb2-502510710ebc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:44.972117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:44.972830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:44.973611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:44.984264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d9110-d99c-4af8-94e5-b68273b094a1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:44.988562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:44.989404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:44.989730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:44.991433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.028910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.029391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.029687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.029972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.050052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:03:45.050786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.053884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.058451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 822/1670] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:45.059649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.103375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bbdf6-2333-4234-a0b9-7566f8014707 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.108693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 849/1671] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:03:45.129806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.131967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.132650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.132940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.133376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.140087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:45.185467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.188422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483394da-38ff-4758-88cc-db11ad4b4d83 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.207318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.211972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.212838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.214612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.215094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.223243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.224449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 823/1672] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:45.225439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.239444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.252767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874708-68aa-4a6e-a3b8-f795a99a4b31 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.282499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.283938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.285154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.289626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.297481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:45.310976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.318248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.409242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 850/1673] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:45 2022] POST /identity/v3/projects => generated 365 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:45.410822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.420262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 824/1674] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:45 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:45.421948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.533668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:45.613066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddc4cb-7ef6-48d9-9ede-b954ae17b8af admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.629632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307613e-f80b-4499-9c7c-a11753a0c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.630535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307613e-f80b-4499-9c7c-a11753a0c1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.630945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307613e-f80b-4499-9c7c-a11753a0c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.631871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307613e-f80b-4499-9c7c-a11753a0c1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.632137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307613e-f80b-4499-9c7c-a11753a0c1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.634972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307613e-f80b-4499-9c7c-a11753a0c1a5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.653096 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f45f7-9788-4a6d-a036-ceeb817b13a7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.684095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 825/1675] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:45 2022] POST /identity/v3/users => generated 282 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:45.685596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117fa5-a194-47d2-b314-44b96e5a9a45 None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.687172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7da79-71b7-4a35-b948-60147c066264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.687632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117fa5-a194-47d2-b314-44b96e5a9a45 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.691415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7da79-71b7-4a35-b948-60147c066264 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.691775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7da79-71b7-4a35-b948-60147c066264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.691999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117fa5-a194-47d2-b314-44b96e5a9a45 None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.693194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7da79-71b7-4a35-b948-60147c066264 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.693740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 826/1676] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 12:03:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:45.694015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7da79-71b7-4a35-b948-60147c066264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.699717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44117fa5-a194-47d2-b314-44b96e5a9a45 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.704255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7da79-71b7-4a35-b948-60147c066264 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.735318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44117fa5-a194-47d2-b314-44b96e5a9a45 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.750290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.751118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.751502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.751878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.752720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.754455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.763335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 851/1677] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:45 2022] POST /identity/v3/users => generated 283 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:45.763335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.764445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.764755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.779602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 827/1678] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:45.779602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.802715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df95d6-5e83-45ab-8fb4-53d89bca09bb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.815599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.816468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.816866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.816972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:45.817896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.818162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.819940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.820276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:45.853185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 828/1679] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:45.854256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.879153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68be745-8912-41b3-836b-59fa063e711a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.893315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.894158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.894556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.894962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.895806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.897528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:45.919680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 829/1680] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:45.920657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:45.945946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340edab-981c-493b-80c2-2ef840c0df22 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:45.972297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:45.973903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:45.975323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:45.975754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:45.979068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:45.984431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:46.015442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 830/1681] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:46.015442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:46.050369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 852/1682] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 12:03:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:46.052233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:46.066321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d27d5-7091-4b28-b0f2-d749a1d83beb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:46.078756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e0234-bb39-486f-957c-09b4ce2189a2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:46.081084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6f463-50ea-4952-8d0b-2b7e94e7613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:46.082002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6f463-50ea-4952-8d0b-2b7e94e7613e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.089056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6f463-50ea-4952-8d0b-2b7e94e7613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.089377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6f463-50ea-4952-8d0b-2b7e94e7613e admin admin] PATH_INFO: `/v3/projects/815d9dcf3ed24baab0bb3bac1037334e/users/1b8bb70858a445f1addc311480ce777f/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.090198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee51326-5923-4f08-8744-f1ebfc8d02a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:46.095960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee51326-5923-4f08-8744-f1ebfc8d02a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.096268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee51326-5923-4f08-8744-f1ebfc8d02a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.096549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee51326-5923-4f08-8744-f1ebfc8d02a2 admin admin] PATH_INFO: `/v3/projects/2af7c3e4e32c4a54840ce9b72bdcf467/users/fbfddbdd89da4a569bca400baf3b4de1/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.147066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee51326-5923-4f08-8744-f1ebfc8d02a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af7c3e4e32c4a54840ce9b72bdcf467, user_id=fbfddbdd89da4a569bca400baf3b4de1, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:46.155861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6f463-50ea-4952-8d0b-2b7e94e7613e admin admin] RBAC: Authorizing `identity:create_grant(project_id=815d9dcf3ed24baab0bb3bac1037334e, user_id=1b8bb70858a445f1addc311480ce777f, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:46.156598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee51326-5923-4f08-8744-f1ebfc8d02a2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:46.158585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6f463-50ea-4952-8d0b-2b7e94e7613e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:46.203530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 853/1683] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:46 2022] PUT /identity/v3/projects/2af7c3e4e32c4a54840ce9b72bdcf467/users/fbfddbdd89da4a569bca400baf3b4de1/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:46.208246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029becb-dac5-4a66-866d-08622e3a6350 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.209063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029becb-dac5-4a66-866d-08622e3a6350 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.209510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029becb-dac5-4a66-866d-08622e3a6350 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.214546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 854/1684] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 12:03:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:46.216255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ce5e-b9ef-466b-833c-fc2eddf30e54 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.217589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ce5e-b9ef-466b-833c-fc2eddf30e54 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.218501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ce5e-b9ef-466b-833c-fc2eddf30e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.219011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 831/1685] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:46 2022] PUT /identity/v3/projects/815d9dcf3ed24baab0bb3bac1037334e/users/1b8bb70858a445f1addc311480ce777f/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:46.220238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.220532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.220816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.290468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:46.293591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:46.307130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6290ce5e-b9ef-466b-833c-fc2eddf30e54 None None] MFA Rules not processed for user `fbfddbdd89da4a569bca400baf3b4de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:46.309185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6290ce5e-b9ef-466b-833c-fc2eddf30e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:46.389560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:46.470921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 855/1686] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:46.478414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.478774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.479024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.516439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 832/1687] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:03:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:46.521943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:46.550303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] MFA Rules not processed for user `1b8bb70858a445f1addc311480ce777f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:46.551739 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:46.703334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:46.752256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8afa43-ee2a-42fc-8527-149c29dbd230 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:46.766342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:46.767310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:46.767914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:46.768331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:46.769180 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:46.771333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:46.792863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 856/1688] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:46.794052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:46.798491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:46.833468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 833/1689] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:46 2022] POST /identity/v3/projects => generated 365 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:46.835105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:46.981311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf94ec-2416-4ba2-ae54-a9138aceae0c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.020122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d502b-732a-4cc6-8ae5-144f80e69b30 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.034757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.035676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.035966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.036247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.036952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.038940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.102383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 834/1690] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:03:46 2022] POST /identity/v3/projects => generated 367 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:47.103814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.122834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.124078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.124462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.124892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.214167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:47.300244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9379d-293c-4e45-bf95-10091b5fb9fe admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.309658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.310819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.313645 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839b0d-334b-4922-a818-0911610105db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.316645 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839b0d-334b-4922-a818-0911610105db admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.317026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839b0d-334b-4922-a818-0911610105db admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.319795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82839b0d-334b-4922-a818-0911610105db admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.320171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82839b0d-334b-4922-a818-0911610105db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.328420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82839b0d-334b-4922-a818-0911610105db admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.346073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 857/1691] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:47.347137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.374785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324164b5-568d-4e45-8d17-879ad4c79a20 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.384843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 835/1692] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:47 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:47.385866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839b0d-334b-4922-a818-0911610105db admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.394561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.398712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.400035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.400430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.404865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.414011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82839b0d-334b-4922-a818-0911610105db admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.417139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.437377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.438087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.438377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.438659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.445246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.455314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.485932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 858/1693] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:03:47 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:03:47.487064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.499209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 836/1694] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:47.500865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.521271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660546a5-b30f-4012-a35a-68b93894b660 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.531718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1873e8-d631-4806-ba79-756411d72a16 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.547177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.547997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.548491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.548881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.549805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.550249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.551723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.555431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.556042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.556494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] PATH_INFO: `/v3/projects/54fcd81caa114927bf8cf092c08584ba/users/9b6803e43e2b46798295c5e92be17b6e/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.580411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 859/1695] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:47.585444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.600329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fcd81caa114927bf8cf092c08584ba, user_id=9b6803e43e2b46798295c5e92be17b6e, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.611697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91258ce2-0371-4246-9b66-15969cb7d523 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.611877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.624060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.625069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.626121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.626558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] PATH_INFO: `/v3/projects/2ee44600bdea4b9cade2f731bde85c43/users/a1403a85e6a6492583d5ddf439aabe4c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.656856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 837/1696] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:47 2022] PUT /identity/v3/projects/54fcd81caa114927bf8cf092c08584ba/users/9b6803e43e2b46798295c5e92be17b6e/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:47.657959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.694464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee44600bdea4b9cade2f731bde85c43, user_id=a1403a85e6a6492583d5ddf439aabe4c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.695374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:47.702032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:47.737956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 860/1697] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:47 2022] PUT /identity/v3/projects/2ee44600bdea4b9cade2f731bde85c43/users/a1403a85e6a6492583d5ddf439aabe4c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:47.739129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:47.865171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d123015-1d12-45fb-b5ea-03ea5c15459c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.976258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48031b14-b935-40a1-86f7-7725f762a8ab admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:47.986667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4f2bd-401d-4bd1-8282-52fa419074a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:47.987916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4f2bd-401d-4bd1-8282-52fa419074a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:47.988510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4f2bd-401d-4bd1-8282-52fa419074a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:47.989017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4f2bd-401d-4bd1-8282-52fa419074a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:47.990322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4f2bd-401d-4bd1-8282-52fa419074a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:47.992262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4f2bd-401d-4bd1-8282-52fa419074a0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:48.018972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 861/1698] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:48.021231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1505-7e2f-44cd-963f-a960e0a9b470 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.026007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1505-7e2f-44cd-963f-a960e0a9b470 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.027378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1505-7e2f-44cd-963f-a960e0a9b470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.043763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:48.044472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.044758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.045033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.045772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:48.047443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:48.077331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 838/1699] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:03:48.089873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:48.108373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ccfa1505-7e2f-44cd-963f-a960e0a9b470 None None] MFA Rules not processed for user `7372171c6dd44f398ca219ad5502a240`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:48.109760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ccfa1505-7e2f-44cd-963f-a960e0a9b470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:48.117368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40cab8-0e1a-419b-9858-3f9ecd10042b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:48.133069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:48.134521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.134808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.135111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] PATH_INFO: `/v3/projects/2ee44600bdea4b9cade2f731bde85c43/users/a1403a85e6a6492583d5ddf439aabe4c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.137371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee44600bdea4b9cade2f731bde85c43, user_id=a1403a85e6a6492583d5ddf439aabe4c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:48.140112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:48.166779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 839/1700] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:48 2022] PUT /identity/v3/projects/2ee44600bdea4b9cade2f731bde85c43/users/a1403a85e6a6492583d5ddf439aabe4c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:48.176534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:48.207209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:48.242608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 862/1701] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:48.244376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.244781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.245252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.279875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] MFA Rules not processed for user `a1403a85e6a6492583d5ddf439aabe4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:48.281528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:48.309248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36932970-7b69-46c6-853e-ddd111030e6f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:48.324791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bb87e-cb8b-4345-b648-6fe2146ea491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:48.325647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bb87e-cb8b-4345-b648-6fe2146ea491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.325931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bb87e-cb8b-4345-b648-6fe2146ea491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.326208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bb87e-cb8b-4345-b648-6fe2146ea491 admin admin] PATH_INFO: `/v3/projects/54fcd81caa114927bf8cf092c08584ba/users/9b6803e43e2b46798295c5e92be17b6e/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.328405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bb87e-cb8b-4345-b648-6fe2146ea491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fcd81caa114927bf8cf092c08584ba, user_id=9b6803e43e2b46798295c5e92be17b6e, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:48.331143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bb87e-cb8b-4345-b648-6fe2146ea491 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:48.365356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 840/1702] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:03:48 2022] PUT /identity/v3/projects/54fcd81caa114927bf8cf092c08584ba/users/9b6803e43e2b46798295c5e92be17b6e/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:48.376072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.376363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.376642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.377788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 863/1703] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:48 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:48.392507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:48.393908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:48.441770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] MFA Rules not processed for user `9b6803e43e2b46798295c5e92be17b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:48.443122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:48.521174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:48.525223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14625a59-0c38-414a-843c-35d36d2969e2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:48.644909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3f59f-b7c6-4750-aeda-654bfbed7632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:48.645586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3f59f-b7c6-4750-aeda-654bfbed7632 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.645879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3f59f-b7c6-4750-aeda-654bfbed7632 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.646160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3f59f-b7c6-4750-aeda-654bfbed7632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.651997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 841/1704] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:48.663295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:48.665632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:48.711219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4747-e7a7-4ca7-a3b9-0263a26ce446 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:48.727440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:48.727910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:48.728363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:48.729370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:48.748034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:48.749845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:48.771671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 842/1705] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:48.796741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:48.831013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3f59f-b7c6-4750-aeda-654bfbed7632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:48.832320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3f59f-b7c6-4750-aeda-654bfbed7632 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:48.863220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 864/1706] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:48 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:49.082553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:49.096265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96fe86-670c-4fc7-a512-5319f1a4d3bd service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:49.105110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:49.105906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:49.106265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:49.106615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:49.281761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:49.284468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:49.335705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 843/1707] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:49.575804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:49.576702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:49.577195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:49.608467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:49.615368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:49.755586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 865/1708] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:03:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:49.806120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:49.915578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e95a7f-d143-42c3-a72b-cae2d0db37b8 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:49.936158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:49.940227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:49.940517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:49.941305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] PATH_INFO: `/v3/users/fbfddbdd89da4a569bca400baf3b4de1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:49.943244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbfddbdd89da4a569bca400baf3b4de1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:49.945829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:50.032996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:50.044355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbfddbdd89da4a569bca400baf3b4de1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:50.045141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbfddbdd89da4a569bca400baf3b4de1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:50.045763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbfddbdd89da4a569bca400baf3b4de1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:50.071797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbfddbdd89da4a569bca400baf3b4de1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:50.107069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:50.108845 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:50.127384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-890d9466-129e-40e8-beb7-643b721f21ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbfddbdd89da4a569bca400baf3b4de1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:50.157291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 844/1709] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:49 2022] DELETE /identity/v3/users/fbfddbdd89da4a569bca400baf3b4de1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:50.159605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:50.159975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:50.161012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:50.240607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:50.256702 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:50.396687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc124c-3510-4861-a7ec-d87b35a6ebe7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:50.471745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 845/1710] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:50.628909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:50.630312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:50.630820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:50.631311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:50.639160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:50.706882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:50.781899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:50.782088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:50.783634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0338fbb-8172-48db-91ab-cb1ae66f320e service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:50.783777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:50.834697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 866/1711] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:03:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 12:03:50.938363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573a871-fc19-4630-b4ff-b4a4e0f7cafc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:50.946378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a021b-a6d9-4814-b2d2-2620be6988fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:50.947108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a021b-a6d9-4814-b2d2-2620be6988fb service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:50.947460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a021b-a6d9-4814-b2d2-2620be6988fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:50.949424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a021b-a6d9-4814-b2d2-2620be6988fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:50.981248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a021b-a6d9-4814-b2d2-2620be6988fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:50.988062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a021b-a6d9-4814-b2d2-2620be6988fb service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:51.050496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 846/1712] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:51.320341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270e8db-b4c1-45ba-873c-14cad54711d0 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:51.321632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270e8db-b4c1-45ba-873c-14cad54711d0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:51.322185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270e8db-b4c1-45ba-873c-14cad54711d0 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:51.324751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 867/1713] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:03:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:51.555635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:51.557395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:51.560748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:51.641641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:51.641641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:51.823278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:51.833573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 847/1714] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:03:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:51.939819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270e8db-b4c1-45ba-873c-14cad54711d0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:51.942307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d270e8db-b4c1-45ba-873c-14cad54711d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:52.027769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270e8db-b4c1-45ba-873c-14cad54711d0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:52.041839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee3869-4e05-4004-858d-08be9274b6db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:52.043844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3869-4e05-4004-858d-08be9274b6db service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:52.044317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3869-4e05-4004-858d-08be9274b6db service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:52.045109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3869-4e05-4004-858d-08be9274b6db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:52.079426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee3869-4e05-4004-858d-08be9274b6db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:52.081244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee3869-4e05-4004-858d-08be9274b6db service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:52.108686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 868/1715] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:54.356017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:54.360638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:54.396687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:54.397567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:54.397999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:54.422152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6de5-80d0-4039-b97a-1dd4ed17cc45 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:54.439855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:54.440990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:54.441436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:54.442648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] PATH_INFO: `/v3/projects/2af7c3e4e32c4a54840ce9b72bdcf467` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:54.451013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af7c3e4e32c4a54840ce9b72bdcf467)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:54.454299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:54.455664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:54.458854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:54.506312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 869/1716] 10.222.0.44 () {58 vars in 1015 bytes} [Mon Jun 6 12:03:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:03:54.587774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af7c3e4e32c4a54840ce9b72bdcf467', 'request_id': 'req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:54.611304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af7c3e4e32c4a54840ce9b72bdcf467', 'request_id': 'req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:54.623982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af7c3e4e32c4a54840ce9b72bdcf467', 'request_id': 'req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:54.637596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:54.639302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:54.652300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] The token cache is being invalidate because project 2af7c3e4e32c4a54840ce9b72bdcf467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:03:54.652651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:54.653357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:54.658093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 848/1717] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:54 2022] DELETE /identity/v3/projects/2af7c3e4e32c4a54840ce9b72bdcf467 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:54.672570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2215f1-88ec-4e0c-b75d-940f453a2403 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:54.681477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:54.683346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:54.800531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:54.801501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:54.801895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:54.802343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:54.808132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:54.828918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:54.830582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8b5c9-9a75-42a1-92f9-546fa6a88f70 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:54.861487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 870/1718] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:54.990329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacbc41-75ea-49ef-9d3c-90b7264ce2ac admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:55.000695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:55.001695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:55.002075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:55.002394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] PATH_INFO: `/v3/users/9b6803e43e2b46798295c5e92be17b6e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:55.016772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6803e43e2b46798295c5e92be17b6e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:55.018745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:55.144399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6803e43e2b46798295c5e92be17b6e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.145844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6803e43e2b46798295c5e92be17b6e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.146679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6803e43e2b46798295c5e92be17b6e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.154779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230fcd-938b-4924-8199-ed1eb5952047 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:55.156373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230fcd-938b-4924-8199-ed1eb5952047 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:55.156834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230fcd-938b-4924-8199-ed1eb5952047 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:55.158444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 871/1719] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:03:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:03:55.177374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6230fcd-938b-4924-8199-ed1eb5952047 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:55.181260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a6230fcd-938b-4924-8199-ed1eb5952047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:55.201325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6803e43e2b46798295c5e92be17b6e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.279457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6230fcd-938b-4924-8199-ed1eb5952047 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:55.287547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bac5adaf-140a-4d0d-84ed-0617a8dbee97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6803e43e2b46798295c5e92be17b6e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.306320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:55.311121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:55.312442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:55.312982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] PATH_INFO: `/v3/users/ac1534ff6fcd4283b36e2c6d4e019085` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:55.317259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1534ff6fcd4283b36e2c6d4e019085)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:55.320037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:55.335669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 849/1720] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:54 2022] DELETE /identity/v3/users/9b6803e43e2b46798295c5e92be17b6e => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:55.459635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:55.461208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:55.461669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:55.479145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:55.510911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1534ff6fcd4283b36e2c6d4e019085'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.517898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1534ff6fcd4283b36e2c6d4e019085'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.525279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] MFA Rules not processed for user `9c11a2d80dcf4533a2b87fb5d85a552c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:55.529908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:55.544367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1534ff6fcd4283b36e2c6d4e019085'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.547722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1534ff6fcd4283b36e2c6d4e019085'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.566652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5c518c53-e5fb-4d71-a6ea-1fbf3fb80133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1534ff6fcd4283b36e2c6d4e019085'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:55.659469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 872/1721] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:55 2022] DELETE /identity/v3/users/ac1534ff6fcd4283b36e2c6d4e019085 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:55.788982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 850/1722] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:03:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:03:55.930568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56482e8b-946b-4d1a-8e4e-67884fe00a7e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:55.931640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56482e8b-946b-4d1a-8e4e-67884fe00a7e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:55.932143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56482e8b-946b-4d1a-8e4e-67884fe00a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:56.017527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-56482e8b-946b-4d1a-8e4e-67884fe00a7e None None] MFA Rules not processed for user `473532c21cad4c3492c8425855993b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:56.029764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-56482e8b-946b-4d1a-8e4e-67884fe00a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:56.068202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:56.069686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:56.079895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:56.112760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6117732-d14c-42ba-b0ff-81ddd5429e2e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:56.288238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 873/1723] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:56.299199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:56.299817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:56.301008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:56.357726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] MFA Rules not processed for user `473532c21cad4c3492c8425855993b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:56.373835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cf1e0-302b-4873-b253-47b7bdd479db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:56.378688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cf1e0-302b-4873-b253-47b7bdd479db service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:56.380881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cf1e0-302b-4873-b253-47b7bdd479db service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:56.380881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cf1e0-302b-4873-b253-47b7bdd479db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:56.381080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:56.413240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 874/1724] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:56.425898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:56.429741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:56.432560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cf1e0-302b-4873-b253-47b7bdd479db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:56.434240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cf1e0-302b-4873-b253-47b7bdd479db service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:56.483714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 851/1725] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:03:56.676232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95481315-48f8-48ae-9db4-f9e3348f350f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:56.703976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:56.704603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:56.704926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:56.705245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:56.738161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:56.740291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:56.779950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 875/1726] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:58.179149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:58.180053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:58.180550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:58.222743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:03:58.228948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:58.428811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 852/1727] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:03:59.134361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:59.136011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:59.353506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:59.354821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:59.382797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb77c6d-d85b-406b-81a3-40f4ab4d651e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:59.395187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:59.396131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:59.396568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:59.396989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:59.422606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:59.423849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:59.441923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383700d-e630-424d-98db-6b4580b6b25c service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:59.456290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:59.457207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:59.457492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:59.457775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] PATH_INFO: `/v3/users/1b8bb70858a445f1addc311480ce777f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:59.469651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8bb70858a445f1addc311480ce777f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:59.483403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:59.486199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 853/1728] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:03:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:03:59.487150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:59.494957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:59.564485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c92a8-7f5a-4e61-ba41-352d9be1b737 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:03:59.578797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:03:59.580799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:59.581087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:59.582462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] PATH_INFO: `/v3/projects/086d355c6b804aefb0b53d2f7848ed0e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:03:59.584659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=086d355c6b804aefb0b53d2f7848ed0e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:03:59.588166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:59.589318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:03:59.606101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8bb70858a445f1addc311480ce777f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.607494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8bb70858a445f1addc311480ce777f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.624645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8bb70858a445f1addc311480ce777f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.627149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8bb70858a445f1addc311480ce777f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.654446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8bb70858a445f1addc311480ce777f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.659476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:03:59.746155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 876/1729] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:03:59 2022] DELETE /identity/v3/users/1b8bb70858a445f1addc311480ce777f => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:59.751325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:03:59.768663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:03:59.777823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086d355c6b804aefb0b53d2f7848ed0e', 'request_id': 'req-34cfc43b-167d-4d7f-a822-4cba949de0a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.819462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086d355c6b804aefb0b53d2f7848ed0e', 'request_id': 'req-34cfc43b-167d-4d7f-a822-4cba949de0a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.858503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086d355c6b804aefb0b53d2f7848ed0e', 'request_id': 'req-34cfc43b-167d-4d7f-a822-4cba949de0a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.921394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] The token cache is being invalidate because project 086d355c6b804aefb0b53d2f7848ed0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:03:59.921692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.922403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-34cfc43b-167d-4d7f-a822-4cba949de0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:03:59.924024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 854/1730] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:59 2022] DELETE /identity/v3/projects/086d355c6b804aefb0b53d2f7848ed0e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:03:59.928867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:03:59.929203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:03:59.929473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:00.048096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:00.049340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:00.133339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27393d-4653-4687-99da-51540734baf3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:00.173296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:00.174947 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:00.175402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:00.175823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] PATH_INFO: `/v3/projects/54fcd81caa114927bf8cf092c08584ba` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:00.191818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54fcd81caa114927bf8cf092c08584ba)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:00.194165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:00.219641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 855/1731] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:00.220594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:00.225661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:00.261707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc2764-e0bf-4e67-9ce5-e428fce5f4d5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:00.291284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:00.293742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:00.301115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:00.301608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] PATH_INFO: `/v3/users/7372171c6dd44f398ca219ad5502a240` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:00.306146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7372171c6dd44f398ca219ad5502a240)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:00.308353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:00.445848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7372171c6dd44f398ca219ad5502a240'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.454255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7372171c6dd44f398ca219ad5502a240'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.455001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7372171c6dd44f398ca219ad5502a240'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.489863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7372171c6dd44f398ca219ad5502a240'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.491795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54fcd81caa114927bf8cf092c08584ba', 'request_id': 'req-142a2d90-6a32-48f4-8aa2-02dbfada5de0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.527495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-15a027e0-331e-4f43-9070-3f4ac9201bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7372171c6dd44f398ca219ad5502a240'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.527690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54fcd81caa114927bf8cf092c08584ba', 'request_id': 'req-142a2d90-6a32-48f4-8aa2-02dbfada5de0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.550280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54fcd81caa114927bf8cf092c08584ba', 'request_id': 'req-142a2d90-6a32-48f4-8aa2-02dbfada5de0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.577635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 856/1732] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:00 2022] DELETE /identity/v3/users/7372171c6dd44f398ca219ad5502a240 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:00.579121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ad6c-9496-4ef3-890f-4cce1b4a8700 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:00.579487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ad6c-9496-4ef3-890f-4cce1b4a8700 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:00.579701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ad6c-9496-4ef3-890f-4cce1b4a8700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:00.594444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] The token cache is being invalidate because project 54fcd81caa114927bf8cf092c08584ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:00.595846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.597613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:00.601023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 877/1733] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:03:59 2022] DELETE /identity/v3/projects/54fcd81caa114927bf8cf092c08584ba => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:00.603286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:00.618695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:00.760586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7660ad6c-9496-4ef3-890f-4cce1b4a8700 None None] MFA Rules not processed for user `6aebb057a324492699141a878793c097`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:00.762074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7660ad6c-9496-4ef3-890f-4cce1b4a8700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:00.799243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:00.802106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:00.852226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a2d90-6a32-48f4-8aa2-02dbfada5de0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:01.006746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:01.009404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.009771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.012441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.019141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:01.128401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 857/1734] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:01.128401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40d54c-e87a-473d-9d8e-ec0e24e18980 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.128401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40d54c-e87a-473d-9d8e-ec0e24e18980 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.128401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40d54c-e87a-473d-9d8e-ec0e24e18980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.176006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7c40d54c-e87a-473d-9d8e-ec0e24e18980 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:01.177345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7c40d54c-e87a-473d-9d8e-ec0e24e18980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:01.201639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:01.204567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:01.213166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 858/1735] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:01.221346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.221690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.221929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.240463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 878/1736] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:01.288534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:01.296699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:01.327507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 859/1737] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:01.328402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:01.359386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a312ad1-d200-4c55-923a-a98b8f6985bb service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:01.371296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:01.372014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.376574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.376936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.381097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:01.381432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:01.386017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:01.459037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 879/1738] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:01.463164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:01.481243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff537a62-8e4c-437b-8c06-243a95514e39 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:01.489447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:01.490649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.491359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.492421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.495468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:01.499082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:01.552192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 860/1739] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:01 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:01.553457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:01.583647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e624e-fada-423a-a817-45720dbfdbb4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:01.786591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:01.787421 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.787710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.787993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.788696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:01.792165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:01.847204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 880/1740] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:01 2022] POST /identity/v3/users => generated 285 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:01.849469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:01.879394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515a4e-a439-44f1-8da7-2da7e2eb0601 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:01.886233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:01.888835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.889359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.889873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.891285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:01.893862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:01.924008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 861/1741] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:01.924446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:01.937503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452def86-d7e8-4c5e-a2eb-3b4a7e42642f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:01.950573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218f658-2f88-4b00-9f7e-f86a363bc105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:01.951297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218f658-2f88-4b00-9f7e-f86a363bc105 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:01.951589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218f658-2f88-4b00-9f7e-f86a363bc105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:01.951864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218f658-2f88-4b00-9f7e-f86a363bc105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:01.952601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218f658-2f88-4b00-9f7e-f86a363bc105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:01.954245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218f658-2f88-4b00-9f7e-f86a363bc105 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:02.005769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:02.005960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 881/1742] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:02.017690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8722e8b-6dbb-41ba-a30e-020f5c7c4576 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:02.036428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77962b68-a1f3-47fb-b04a-da90e90c9d78 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:02.036718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77962b68-a1f3-47fb-b04a-da90e90c9d78 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:02.037000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77962b68-a1f3-47fb-b04a-da90e90c9d78 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:02.038674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 882/1743] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 12:04:02.048011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d5936-7ca6-4682-a403-473d7e3c8003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:02.049329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d5936-7ca6-4682-a403-473d7e3c8003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:02.049838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d5936-7ca6-4682-a403-473d7e3c8003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:02.050330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d5936-7ca6-4682-a403-473d7e3c8003 admin admin] PATH_INFO: `/v3/projects/8ed02ba17cf84e7fa8fa5a536793e821/users/b5d2289b7acc477c987cccc3d6469635/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:02.069300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:02.069586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:02.069867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:02.095764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d5936-7ca6-4682-a403-473d7e3c8003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed02ba17cf84e7fa8fa5a536793e821, user_id=b5d2289b7acc477c987cccc3d6469635, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:02.098343 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d5936-7ca6-4682-a403-473d7e3c8003 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:02.142084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 862/1744] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:02 2022] PUT /identity/v3/projects/8ed02ba17cf84e7fa8fa5a536793e821/users/b5d2289b7acc477c987cccc3d6469635/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:02.143872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:02.144418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:02.144923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:02.145120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:02.145615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:02.195800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] MFA Rules not processed for user `b5d2289b7acc477c987cccc3d6469635`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:02.197926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:02.200601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:02.369453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 863/1745] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:02.370901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:02.413685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 883/1746] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:04:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:02.482058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:02.483780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:02.500979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:02.522576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02b8cf-ae5d-42c1-b380-6edb2ea4878e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:02.549050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:02.550024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:02.550382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:02.550793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:02.610299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b0a0-1fb5-4620-87ff-1c7fe09ab35d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:02.656724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:02.657763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:02.659789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:02.660295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:02.666265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:02.674947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:02.712295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:02.714367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:02.754078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 864/1747] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:02 2022] POST /identity/v3/projects => generated 369 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:02.755107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:02.775556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 884/1748] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:03.059945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:03.081790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bcedd-7f2a-4120-ba13-2696900caa85 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:03.108965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904dcf7-50b3-499e-92c3-1c1039449840 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:03.149501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:03.155429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:03.155740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:03.156048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:03.160093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:03.172907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:03.282270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 865/1749] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:02 2022] POST /identity/v3/users => generated 291 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:03.286865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:03.319340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:03.320019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:03.320308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:03.320666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:03.354868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0ba07-e892-487b-8a34-2d77a00a17b5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:03.374091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:03.375143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c19f1-63ca-4154-a1c6-0a3b93aa9014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:03.377404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:03.378137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c19f1-63ca-4154-a1c6-0a3b93aa9014 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:03.378541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c19f1-63ca-4154-a1c6-0a3b93aa9014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:03.379676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c19f1-63ca-4154-a1c6-0a3b93aa9014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:03.382801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c19f1-63ca-4154-a1c6-0a3b93aa9014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:03.385738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c19f1-63ca-4154-a1c6-0a3b93aa9014 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:03.425606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 885/1750] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:03.436394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:03.441400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 866/1751] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:03.466811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1becd1b-af7a-44c3-ad81-9bd5c1dfae6d service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:03.468251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:03.468556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:03.468837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:03.492336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db965649-b1b5-4e51-a86c-c406d4ed8f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:03.493664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db965649-b1b5-4e51-a86c-c406d4ed8f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:03.493996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db965649-b1b5-4e51-a86c-c406d4ed8f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:03.497869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db965649-b1b5-4e51-a86c-c406d4ed8f02 admin admin] PATH_INFO: `/v3/projects/f4feacc49b2b4149958d1983926ac71a/users/a888d9786c8948bfb5fe77e7ce0fc26b/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:03.612232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db965649-b1b5-4e51-a86c-c406d4ed8f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4feacc49b2b4149958d1983926ac71a, user_id=a888d9786c8948bfb5fe77e7ce0fc26b, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:03.621343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db965649-b1b5-4e51-a86c-c406d4ed8f02 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:03.621482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] MFA Rules not processed for user `6c904a413f6a4ba49686debac817c70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:03.621482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:03.700285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 886/1752] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:03 2022] PUT /identity/v3/projects/f4feacc49b2b4149958d1983926ac71a/users/a888d9786c8948bfb5fe77e7ce0fc26b/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:03.700285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:03.700285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:03.700285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:03.761417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:03.778031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] MFA Rules not processed for user `473532c21cad4c3492c8425855993b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:03.779311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:03.881701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 867/1753] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:03.881701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:03.904721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed0452-399d-49ea-807f-810e2e4b857a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:03.958142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:03.975562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 887/1754] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:03.980799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:04.134536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:04.139094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.139808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.141562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.143936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:04.150797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:04.201330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 868/1755] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:04.204822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:04.214647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:04.276903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875f59c-d947-4991-8808-47192229d17b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:04.295220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:04.296182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.299384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.299555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] PATH_INFO: `/v3/projects/c2dd1404952a426cab6d3df84da39983` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.301270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9294f-81db-49f0-8eba-c54aac827cba None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:04.302114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2dd1404952a426cab6d3df84da39983)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:04.312986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:04.315076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:04.315850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.317153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.318477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.454801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:04.456847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:04.481307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2dd1404952a426cab6d3df84da39983', 'request_id': 'req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:04.486048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 888/1756] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:04.488978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:04.502642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2dd1404952a426cab6d3df84da39983', 'request_id': 'req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:04.517911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576d630-0005-4cf1-af1f-c01a2dbe5458 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:04.542207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2dd1404952a426cab6d3df84da39983', 'request_id': 'req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:04.588093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] The token cache is being invalidate because project c2dd1404952a426cab6d3df84da39983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:04.590913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:04.592849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-73043ff5-5d9a-4ec4-9a2e-6c1e2c6ac774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:04.600060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 869/1757] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:04 2022] DELETE /identity/v3/projects/c2dd1404952a426cab6d3df84da39983 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:04.617644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2f893-6717-4b88-b4d5-5e2b5280e76f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.617644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2f893-6717-4b88-b4d5-5e2b5280e76f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.617644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2f893-6717-4b88-b4d5-5e2b5280e76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.680041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:04.682750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bbd2f893-6717-4b88-b4d5-5e2b5280e76f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:04.687668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bbd2f893-6717-4b88-b4d5-5e2b5280e76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:04.709922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439debc-e442-449c-88bc-926cddaa1f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:04.717372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439debc-e442-449c-88bc-926cddaa1f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.717722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439debc-e442-449c-88bc-926cddaa1f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.719323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439debc-e442-449c-88bc-926cddaa1f9c admin admin] PATH_INFO: `/v3/projects/f4feacc49b2b4149958d1983926ac71a/users/a888d9786c8948bfb5fe77e7ce0fc26b/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.719671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 870/1758] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:04.721243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.721628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.721973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.731251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439debc-e442-449c-88bc-926cddaa1f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4feacc49b2b4149958d1983926ac71a, user_id=a888d9786c8948bfb5fe77e7ce0fc26b, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:04.741325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439debc-e442-449c-88bc-926cddaa1f9c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:04.775667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 889/1759] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:04 2022] PUT /identity/v3/projects/f4feacc49b2b4149958d1983926ac71a/users/a888d9786c8948bfb5fe77e7ce0fc26b/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:04.784570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:04.786927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:04.790698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.791193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.796355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.933394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 871/1760] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:04.937970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:04.949823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd798b8b-04f5-4eb3-926a-97c08ad191de None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:04.961994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] MFA Rules not processed for user `a888d9786c8948bfb5fe77e7ce0fc26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:04.965713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:04.966425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:04.966720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:04.968151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:04.973130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:04.974086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:04.974419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:04.976656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:05.036347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 872/1761] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:05.040538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:05.084108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f7ca6-e6f1-4ca5-9242-295abdd99980 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:05.111025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:05.111954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:05.112260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:05.112559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:05.113274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:05.120408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:05.189872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:05.208536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 873/1762] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:05 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:05.210240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:05.221646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 890/1763] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:04 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:05.230330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:05.232831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:05.423462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e5de-7229-45bf-9cdd-1f505d6878c7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:05.431788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc971b9-bb52-4da3-8161-13bcfbf0ae4e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:05.445996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:05.447256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:05.447550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:05.447835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:05.448546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:05.456662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:05.514604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 874/1764] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:05 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:05.522135 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:05.527521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:05.569940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1368c60-df50-4705-823e-b0935f917554 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:05.582754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:05.584849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:05.587301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:05.587622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] PATH_INFO: `/v3/projects/815d9dcf3ed24baab0bb3bac1037334e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:05.590576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815d9dcf3ed24baab0bb3bac1037334e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:05.594723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:05.636619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:05.639316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:05.640140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:05.641753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:05.735132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815d9dcf3ed24baab0bb3bac1037334e', 'request_id': 'req-56e2cb6f-ea22-4e98-89da-21369e501fa5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:05.751481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815d9dcf3ed24baab0bb3bac1037334e', 'request_id': 'req-56e2cb6f-ea22-4e98-89da-21369e501fa5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:05.771093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815d9dcf3ed24baab0bb3bac1037334e', 'request_id': 'req-56e2cb6f-ea22-4e98-89da-21369e501fa5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:05.779251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:05.802238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] The token cache is being invalidate because project 815d9dcf3ed24baab0bb3bac1037334e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:05.803757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:05.805912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:05.808906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 875/1765] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:05 2022] DELETE /identity/v3/projects/815d9dcf3ed24baab0bb3bac1037334e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:05.815047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:05.819387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:05.874395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:05.878431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:05.908417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 891/1766] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:05 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:05.909644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:05.951341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03aff-e001-496f-96e0-f3eec7355663 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:06.060430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2cb6f-ea22-4e98-89da-21369e501fa5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:06.075623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:06.076337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:06.076634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:06.076910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:06.083670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:06.085353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:06.123181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 876/1767] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:06.124461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:06.152854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343a916-8faf-439e-9462-4bd7ea6d0d25 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:06.155200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:06.156038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:06.156329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:06.157149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] PATH_INFO: `/v3/users/a1403a85e6a6492583d5ddf439aabe4c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:06.159202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1403a85e6a6492583d5ddf439aabe4c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:06.162542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:06.167307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:06.167999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:06.168289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:06.168573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:06.169308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:06.173887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:06.176764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:06.205995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 877/1768] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:06.212700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:06.242434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55633d2a-ef25-4df5-8af7-c2942d47ef67 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:06.264510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c20d6d-63b4-4f60-9a52-a1bab8d0ca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:06.267126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20d6d-63b4-4f60-9a52-a1bab8d0ca80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:06.267643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20d6d-63b4-4f60-9a52-a1bab8d0ca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:06.268144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20d6d-63b4-4f60-9a52-a1bab8d0ca80 admin admin] PATH_INFO: `/v3/projects/38d9c6e1f3de4047a6bcb863e0fcf247/users/4e5cc302d3854d93bc19453be059b096/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:06.306307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1403a85e6a6492583d5ddf439aabe4c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:06.308059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1403a85e6a6492583d5ddf439aabe4c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:06.315608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c20d6d-63b4-4f60-9a52-a1bab8d0ca80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d9c6e1f3de4047a6bcb863e0fcf247, user_id=4e5cc302d3854d93bc19453be059b096, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:06.320666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c20d6d-63b4-4f60-9a52-a1bab8d0ca80 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:06.324582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1403a85e6a6492583d5ddf439aabe4c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:06.327241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1403a85e6a6492583d5ddf439aabe4c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:06.356427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1403a85e6a6492583d5ddf439aabe4c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:06.386682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 878/1769] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:06 2022] PUT /identity/v3/projects/38d9c6e1f3de4047a6bcb863e0fcf247/users/4e5cc302d3854d93bc19453be059b096/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:06.389493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:06.389877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:06.391785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:06.434977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] MFA Rules not processed for user `4e5cc302d3854d93bc19453be059b096`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:06.436902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:06.489505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 892/1770] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:05 2022] DELETE /identity/v3/users/a1403a85e6a6492583d5ddf439aabe4c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:06.639027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 879/1771] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:06.639755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:06.641334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:06.863670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:06.936170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fbc73-76d7-4975-b47b-34a9b92f9156 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:06.954170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:06.964766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:06.966494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:06.969053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:06.970355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:06.978005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:07.072889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 893/1772] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:06 2022] POST /identity/v3/projects => generated 363 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:07.076658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:07.254525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c2a7f-4282-431e-85f4-95db5d5c73fa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:07.416663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:07.417372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:07.417660 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:07.417946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:07.418646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:07.420635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:07.461216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:07.484353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 880/1773] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:07 2022] POST /identity/v3/users => generated 288 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:07.484538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:07.527046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39d43b-bb2b-40f1-8f8c-6bcd618b14fe admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:07.546140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:07.547716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:07.548209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:07.548951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:07.550595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:07.556827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:07.603140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 894/1774] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:07.605068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:07.624391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf998-8a8c-4c33-9488-5a4388b241f3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:07.651185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:07.652744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:07.653088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:07.653477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] PATH_INFO: `/v3/projects/17bde3f179d9453c9f2efd4367db3a42/users/d948f9a22fe948d091397711b9ff3b28/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:07.691760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bde3f179d9453c9f2efd4367db3a42, user_id=d948f9a22fe948d091397711b9ff3b28, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:07.695008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:07.750361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 881/1775] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:07 2022] PUT /identity/v3/projects/17bde3f179d9453c9f2efd4367db3a42/users/d948f9a22fe948d091397711b9ff3b28/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:07.754358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:07.771161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1130e8-8a38-4986-918e-9bae9995d1a2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:07.974420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320677ea-1c34-4ac1-98cf-d7e109f538bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:07.982535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320677ea-1c34-4ac1-98cf-d7e109f538bc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:07.983168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320677ea-1c34-4ac1-98cf-d7e109f538bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:07.983663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320677ea-1c34-4ac1-98cf-d7e109f538bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:07.984866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320677ea-1c34-4ac1-98cf-d7e109f538bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:07.986822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320677ea-1c34-4ac1-98cf-d7e109f538bc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:08.002298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 895/1776] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:08.006490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:08.024816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4eb85-41a4-4741-848a-b4c0b978ef61 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:08.032695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:08.034898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:08.035187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:08.035829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] PATH_INFO: `/v3/projects/17bde3f179d9453c9f2efd4367db3a42/users/d948f9a22fe948d091397711b9ff3b28/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:08.039055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bde3f179d9453c9f2efd4367db3a42, user_id=d948f9a22fe948d091397711b9ff3b28, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:08.050239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:08.080983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 882/1777] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:08 2022] PUT /identity/v3/projects/17bde3f179d9453c9f2efd4367db3a42/users/d948f9a22fe948d091397711b9ff3b28/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:08.092726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:08.093011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:08.093294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:08.170004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] MFA Rules not processed for user `d948f9a22fe948d091397711b9ff3b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:08.171335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:08.537495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 896/1778] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:08 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:08.547513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:08.549529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:08.809228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d3ac-f54b-4976-ba65-c4f1d4241c1b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:08.817732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:08.820596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:08.821331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:08.822879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:08.855750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:08.876234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:08.913662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 883/1779] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:08 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:09.192657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:09.220957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5967fb6-8af0-45c9-852f-f637a02c1a5a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:09.237905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45397c-a4aa-4dea-a6ed-be2371ce172e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:09.238957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45397c-a4aa-4dea-a6ed-be2371ce172e service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:09.239385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45397c-a4aa-4dea-a6ed-be2371ce172e service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:09.239818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45397c-a4aa-4dea-a6ed-be2371ce172e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:09.424280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:09.459367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45397c-a4aa-4dea-a6ed-be2371ce172e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:09.462582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45397c-a4aa-4dea-a6ed-be2371ce172e service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:09.488808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 897/1780] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:09.641622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:09.644704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:09.705991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:09.706545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:09.712045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:09.763087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] MFA Rules not processed for user `c8bda0c079054e198ffeca4c0ac18bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:09.766321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:09.875896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d181f6f-6c79-4b97-9c55-8db4809186e2 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:09.890381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:09.891725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:09.892029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:09.892318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] PATH_INFO: `/v3/users/b5d2289b7acc477c987cccc3d6469635` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:09.894615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d2289b7acc477c987cccc3d6469635)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:09.897930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:09.928551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 898/1781] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:09.943351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:09.946812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.025046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d2289b7acc477c987cccc3d6469635'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.026532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d2289b7acc477c987cccc3d6469635'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.029552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d2289b7acc477c987cccc3d6469635'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.051832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d2289b7acc477c987cccc3d6469635'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.070778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:10.083221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d2289b7acc477c987cccc3d6469635'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.112381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 884/1782] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:09 2022] DELETE /identity/v3/users/b5d2289b7acc477c987cccc3d6469635 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:10.113761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:10.123570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.136592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c48c4c-9053-4fbc-978c-45a6353f575e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:10.257215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:10.258780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.259322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.259834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.264214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.269653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d189c-1351-42b4-970e-50d9cf797f2d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:10.276371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:10.277439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.277909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.278964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] PATH_INFO: `/v3/projects/2ee44600bdea4b9cade2f731bde85c43` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.280044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee44600bdea4b9cade2f731bde85c43)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:10.282563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:10.292128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:10.373195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:10.374875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29176f27-6aa2-4c6d-bd35-41502448ce10 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:10.394356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee44600bdea4b9cade2f731bde85c43', 'request_id': 'req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.402705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee44600bdea4b9cade2f731bde85c43', 'request_id': 'req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.410674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee44600bdea4b9cade2f731bde85c43', 'request_id': 'req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.413820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 899/1783] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:10.418076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.418385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.418672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.431153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] The token cache is being invalidate because project 2ee44600bdea4b9cade2f731bde85c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:10.431798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.432747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ceca9cf4-0c8c-4233-8d38-ad110d363dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:10.435125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 885/1784] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:10 2022] DELETE /identity/v3/projects/2ee44600bdea4b9cade2f731bde85c43 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:10.439658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f6917-4806-4b45-b757-b97ed53c6753 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.440191 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f6917-4806-4b45-b757-b97ed53c6753 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.440873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f6917-4806-4b45-b757-b97ed53c6753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.466588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:10.466773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-125f6917-4806-4b45-b757-b97ed53c6753 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:10.467868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.468698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-125f6917-4806-4b45-b757-b97ed53c6753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.622459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 886/1785] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:10.623436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 900/1786] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:10.628929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.629344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.629725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.659053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:10.660471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.672863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 887/1787] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:10.673227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:10.674413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:10.675167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:10.689213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c16931-0307-44f8-820d-2fcb06957b68 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:10.694632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:10.695628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.696727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.697167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.698238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:10.699214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:10.700968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:10.704324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:10.730535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 888/1788] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:10.732661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:10.755735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fa328-cf7c-4986-bb57-21ee6a928a0f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:10.771489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ddb75-df83-4c93-9a28-10e85c01e2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:10.772474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ddb75-df83-4c93-9a28-10e85c01e2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.774067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ddb75-df83-4c93-9a28-10e85c01e2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.774568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ddb75-df83-4c93-9a28-10e85c01e2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.776584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ddb75-df83-4c93-9a28-10e85c01e2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:10.779972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ddb75-df83-4c93-9a28-10e85c01e2e1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:10.885075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833367-4aef-4365-9f76-4932e4533ef8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:10.901943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:10.907117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.908080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.911297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.928457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 889/1789] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:10 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:10.936747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:10.940113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:10.940994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:10.957606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:10.961292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:10.997708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 901/1790] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:10.999031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] MFA Rules not processed for user `6c904a413f6a4ba49686debac817c70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:10.999765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:11.001573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:11.067882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f1ce-c668-439a-9325-fe5a60ceff3c service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:11.254971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 890/1791] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:11.260780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:11.277092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:11.294423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:11.294795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:11.295259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:11.296061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:11.299371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:11.376028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 902/1792] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:10 2022] POST /identity/v3/users => generated 287 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:11.377051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:11.449869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d7a8e-5bce-445f-ad7f-803a7d667829 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:11.460397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:11.462450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:11.462950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:11.463408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:11.466620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:11.475521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:11.493847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 891/1793] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:11.497334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:11.511747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf108-c966-48d8-a006-98770f0d0fe9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:11.534003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c62c-48ba-40c2-82ae-b5972aa4b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:11.534725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c62c-48ba-40c2-82ae-b5972aa4b09d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:11.535044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c62c-48ba-40c2-82ae-b5972aa4b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:11.535332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c62c-48ba-40c2-82ae-b5972aa4b09d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:11.536080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c62c-48ba-40c2-82ae-b5972aa4b09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:11.537735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c62c-48ba-40c2-82ae-b5972aa4b09d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:11.583685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:11.585798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 903/1794] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:11.606359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba88a-394c-47bf-8ec2-6d3173953a29 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:11.614941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:11.615996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:11.616293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:11.616585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] PATH_INFO: `/v3/projects/d4ed0448570344758cc1f516eef58ac9/users/eda2e30bdf5f430aaa71818dcf02cb34/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:11.653977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ed0448570344758cc1f516eef58ac9, user_id=eda2e30bdf5f430aaa71818dcf02cb34, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:11.657179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:11.702413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 892/1795] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:11 2022] PUT /identity/v3/projects/d4ed0448570344758cc1f516eef58ac9/users/eda2e30bdf5f430aaa71818dcf02cb34/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:11.703642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:11.706251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:11.706813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:11.761491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:11.766809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:11.920413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 904/1796] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:11.969856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:11.975138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:12.122970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:12.124894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:12.170823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b586cec-a2d0-4a21-af37-31025e3922a5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:12.190297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:12.193452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:12.193777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:12.194356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] PATH_INFO: `/v3/projects/8ed02ba17cf84e7fa8fa5a536793e821` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:12.201253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed02ba17cf84e7fa8fa5a536793e821)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:12.204137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:12.292916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19b216-2128-4d0b-a548-8bd3ff99f4d7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:12.308149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:12.308835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:12.309128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:12.309414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:12.358534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:12.360060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:12.396205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 893/1797] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:12.416021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed02ba17cf84e7fa8fa5a536793e821', 'request_id': 'req-016d4d9b-0979-4eaf-a318-684e58ee87e0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.438569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed02ba17cf84e7fa8fa5a536793e821', 'request_id': 'req-016d4d9b-0979-4eaf-a318-684e58ee87e0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.450713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed02ba17cf84e7fa8fa5a536793e821', 'request_id': 'req-016d4d9b-0979-4eaf-a318-684e58ee87e0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.471764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] The token cache is being invalidate because project 8ed02ba17cf84e7fa8fa5a536793e821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:12.472624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.472746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.479036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 905/1798] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:12 2022] DELETE /identity/v3/projects/8ed02ba17cf84e7fa8fa5a536793e821 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:12.482186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:12.501075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79550e8d-485b-4328-96f1-249d33d49217 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:12.632719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:12.654019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:12.655089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:12.655377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:12.655659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] PATH_INFO: `/v3/users/a888d9786c8948bfb5fe77e7ce0fc26b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:12.657824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a888d9786c8948bfb5fe77e7ce0fc26b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:12.662946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:12.761266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a888d9786c8948bfb5fe77e7ce0fc26b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.762306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a888d9786c8948bfb5fe77e7ce0fc26b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.763155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a888d9786c8948bfb5fe77e7ce0fc26b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.775102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a888d9786c8948bfb5fe77e7ce0fc26b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.794388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-682580cb-0b80-44e9-8928-9cecd9424c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a888d9786c8948bfb5fe77e7ce0fc26b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:12.862038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 894/1799] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:12 2022] DELETE /identity/v3/users/a888d9786c8948bfb5fe77e7ce0fc26b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:13.198539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:13.200611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:13.218042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d4d9b-0979-4eaf-a318-684e58ee87e0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:13.280717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:13.294061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:13.295841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:13.296419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:13.296901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] PATH_INFO: `/v3/users/4e5cc302d3854d93bc19453be059b096` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:13.298448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5cc302d3854d93bc19453be059b096)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:13.300772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:13.370883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5cc302d3854d93bc19453be059b096'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:13.371783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5cc302d3854d93bc19453be059b096'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:13.386388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5cc302d3854d93bc19453be059b096'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:13.387713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5cc302d3854d93bc19453be059b096'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:13.401367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5cc302d3854d93bc19453be059b096'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:13.433393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 906/1800] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:13 2022] DELETE /identity/v3/users/4e5cc302d3854d93bc19453be059b096 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:13.438076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:13.438428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:13.438719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:13.459620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:13.461410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:13.555099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 895/1801] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:13.620847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:13.624888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:13.782722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e41b1-d9ab-45b2-bfcc-4a82e89dd54d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:13.788906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d470551-282f-4563-85f2-e8042af6ec46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:13.789963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d470551-282f-4563-85f2-e8042af6ec46 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:13.790625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d470551-282f-4563-85f2-e8042af6ec46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:13.791084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d470551-282f-4563-85f2-e8042af6ec46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:13.809755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d470551-282f-4563-85f2-e8042af6ec46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:13.813559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d470551-282f-4563-85f2-e8042af6ec46 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:13.864893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 907/1802] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:14.203386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:14.204938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:14.229556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a514a1-f1bc-402f-8f82-4cc1c98c2544 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:14.256153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:14.257388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:14.257858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:14.258314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] PATH_INFO: `/v3/projects/f4feacc49b2b4149958d1983926ac71a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:14.260999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4feacc49b2b4149958d1983926ac71a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:14.267899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:14.375300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4feacc49b2b4149958d1983926ac71a', 'request_id': 'req-427b3d10-5656-495a-a73a-657b74339729'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:14.393685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4feacc49b2b4149958d1983926ac71a', 'request_id': 'req-427b3d10-5656-495a-a73a-657b74339729'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:14.396921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:14.407314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4feacc49b2b4149958d1983926ac71a', 'request_id': 'req-427b3d10-5656-495a-a73a-657b74339729'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:14.449934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] The token cache is being invalidate because project f4feacc49b2b4149958d1983926ac71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:14.450586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:14.451607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-427b3d10-5656-495a-a73a-657b74339729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:14.456057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 896/1803] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:14 2022] DELETE /identity/v3/projects/f4feacc49b2b4149958d1983926ac71a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:14.463575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:14.463990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:14.464060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:14.501262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:14.502608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:14.717293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 908/1804] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:14.727092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525306bf-42f4-4510-bfa5-8391c73b9e61 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:14.727413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525306bf-42f4-4510-bfa5-8391c73b9e61 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:14.727787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525306bf-42f4-4510-bfa5-8391c73b9e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:14.767161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-525306bf-42f4-4510-bfa5-8391c73b9e61 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:14.770071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-525306bf-42f4-4510-bfa5-8391c73b9e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:14.792576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 897/1805] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:14.797465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:14.822987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6345fa0-cdfc-40ad-9c5a-fb3708e97326 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:14.838316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:14.839296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:14.840153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:14.840612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:14.841620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:14.842116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:14.844617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:14.877747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b356-3987-400f-b88e-c9170b12827f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:14.877747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b356-3987-400f-b88e-c9170b12827f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:14.877747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b356-3987-400f-b88e-c9170b12827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:14.878601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 909/1806] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:14.879722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:14.906555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106e379-63ba-46ab-b4d7-fe335a9686e1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:14.920124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:14.921138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:14.928436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e961b356-3987-400f-b88e-c9170b12827f None None] MFA Rules not processed for user `6aebb057a324492699141a878793c097`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:14.929676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e961b356-3987-400f-b88e-c9170b12827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:14.933976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:14.937710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:14.938676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:14.951028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:14.965379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:15.024832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 910/1807] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:14 2022] POST /identity/v3/projects => generated 401 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:15.034548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:15.098206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 898/1808] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:15.187152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcafef1-17d9-48bd-906f-c328eb15e10e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:15.199683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:15.200561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.200924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.201298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.202202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:15.214028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:15.257519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 911/1809] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:15 2022] POST /identity/v3/users => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:15.257934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b356-3987-400f-b88e-c9170b12827f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:15.280813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b356-3987-400f-b88e-c9170b12827f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:15.289005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:15.289651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.290216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.290785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.291735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:15.293369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:15.316977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 899/1810] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:15.322086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:15.341142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08d477-9c92-4d55-aaf1-3d31b3241f0f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:15.354435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdaa06-4c32-46f2-9281-e310c3fe61a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:15.355448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdaa06-4c32-46f2-9281-e310c3fe61a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.355837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdaa06-4c32-46f2-9281-e310c3fe61a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.357239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdaa06-4c32-46f2-9281-e310c3fe61a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.359705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdaa06-4c32-46f2-9281-e310c3fe61a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:15.362076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdaa06-4c32-46f2-9281-e310c3fe61a8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:15.376716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 912/1811] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:15.380720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:15.402723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca99d-1649-4ff9-a2a3-d8a56ed9a7df admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:15.418375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:15.419314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.419599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.419878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] PATH_INFO: `/v3/projects/fc3006407e034228af618bcf92776f07/users/a213b8a65ff9453aa7feb15db373fbf2/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.444060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3006407e034228af618bcf92776f07, user_id=a213b8a65ff9453aa7feb15db373fbf2, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:15.450490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:15.489841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 900/1812] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:15 2022] PUT /identity/v3/projects/fc3006407e034228af618bcf92776f07/users/a213b8a65ff9453aa7feb15db373fbf2/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:15.504062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.504617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.504617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.548622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] MFA Rules not processed for user `a213b8a65ff9453aa7feb15db373fbf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:15.550446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:15.694001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 913/1813] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:15.694514 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:15.770924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290dcc0d-503f-42d9-912e-c5971e8bce25 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:15.785061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:15.786032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.786320 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.786603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.787398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:15.789346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:15.847284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 901/1814] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:15 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:15.851799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:15.868720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd629a8-c6b0-48dc-80ad-bc6fac530fa5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:15.975158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:15.990358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:15.991089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:15.992668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:15.993094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:15.994658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:15.998545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.050897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 914/1815] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:15 2022] POST /identity/v3/users => generated 307 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:16.054628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.070673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a19c05-a4e0-4a61-8f00-edb64196f3fc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.087684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:16.089045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.089354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.091621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.092441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:16.094120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.111421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 902/1816] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:16.115324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.128414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd51df3-0416-41ea-ae86-b9ecc764aa50 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.134416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:16.136364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.138284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.138992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] PATH_INFO: `/v3/projects/c4cdb966224b4f0f82ceab9047241199/users/5c5daf6fb5794fc9a3b8471b39cc79cd/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.158361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cdb966224b4f0f82ceab9047241199, user_id=5c5daf6fb5794fc9a3b8471b39cc79cd, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:16.163439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.187019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 915/1817] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:16 2022] PUT /identity/v3/projects/c4cdb966224b4f0f82ceab9047241199/users/5c5daf6fb5794fc9a3b8471b39cc79cd/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:16.191138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.204263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b7dd-6907-4413-aabf-de190b9a5517 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.239386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:16.278730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1033-ee6d-4acf-96d5-ccfdc58777c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:16.280250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1033-ee6d-4acf-96d5-ccfdc58777c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.280727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1033-ee6d-4acf-96d5-ccfdc58777c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.281193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1033-ee6d-4acf-96d5-ccfdc58777c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.282140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1033-ee6d-4acf-96d5-ccfdc58777c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:16.284754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1033-ee6d-4acf-96d5-ccfdc58777c0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.299163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 903/1818] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:16.303069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.327166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f944b3-a702-492a-839d-ed754f6034e0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.332594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:16.333601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.333959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.335534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] PATH_INFO: `/v3/projects/c4cdb966224b4f0f82ceab9047241199/users/5c5daf6fb5794fc9a3b8471b39cc79cd/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.337316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cdb966224b4f0f82ceab9047241199, user_id=5c5daf6fb5794fc9a3b8471b39cc79cd, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:16.340816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.378121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 916/1819] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:16 2022] PUT /identity/v3/projects/c4cdb966224b4f0f82ceab9047241199/users/5c5daf6fb5794fc9a3b8471b39cc79cd/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:16.394455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.394758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.395074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.453652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] MFA Rules not processed for user `5c5daf6fb5794fc9a3b8471b39cc79cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:16.455500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:16.585152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 904/1820] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:16.596691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.598698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:16.680066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.681171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f68b-1e19-4220-b90a-1f24fe1f023a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.682355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:16.689746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:16.691221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.691221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.691851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.716893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:16.723373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.753034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 917/1821] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:16.796448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9af92a-6f3e-4198-844d-2e7b11f4e973 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.810454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:16.810454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:16.810454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:16.810454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] PATH_INFO: `/v3/projects/38d9c6e1f3de4047a6bcb863e0fcf247` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:16.814194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d9c6e1f3de4047a6bcb863e0fcf247)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:16.820530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:16.866978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.896014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24369cb3-87ef-46c4-9a04-0b2320d8e655 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:16.934651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d9c6e1f3de4047a6bcb863e0fcf247', 'request_id': 'req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:16.936449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:16.949188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d9c6e1f3de4047a6bcb863e0fcf247', 'request_id': 'req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:16.960322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d9c6e1f3de4047a6bcb863e0fcf247', 'request_id': 'req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:16.983666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] The token cache is being invalidate because project 38d9c6e1f3de4047a6bcb863e0fcf247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:16.984061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:16.984824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:16.990674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 905/1822] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:16 2022] DELETE /identity/v3/projects/38d9c6e1f3de4047a6bcb863e0fcf247 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:16.991818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:16.994295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:17.004759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:17.005437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.005735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.006022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.007990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:17.173424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:17.187791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a664068-1430-46df-b769-278422a606e7 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:17.211017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d62ec-5710-4cf8-b6b6-39bc08567ef2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:17.240982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 918/1823] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:17.244164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8cbf-db80-43f0-921f-747117dee516 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.247142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8cbf-db80-43f0-921f-747117dee516 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.247460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8cbf-db80-43f0-921f-747117dee516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.248157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:17.249341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.249731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.250284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] PATH_INFO: `/v3/users/d948f9a22fe948d091397711b9ff3b28` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.252290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d948f9a22fe948d091397711b9ff3b28)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:17.254404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:17.260474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:17.350141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-914a8cbf-db80-43f0-921f-747117dee516 None None] MFA Rules not processed for user `eda2e30bdf5f430aaa71818dcf02cb34`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:17.351422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-914a8cbf-db80-43f0-921f-747117dee516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:17.377812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd948f9a22fe948d091397711b9ff3b28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:17.378632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd948f9a22fe948d091397711b9ff3b28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:17.379693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd948f9a22fe948d091397711b9ff3b28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:17.402932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd948f9a22fe948d091397711b9ff3b28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:17.434641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2af7b44b-d91f-4edd-b314-29e794a1e1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd948f9a22fe948d091397711b9ff3b28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:17.459638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 906/1824] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:16 2022] DELETE /identity/v3/users/d948f9a22fe948d091397711b9ff3b28 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:17.461466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ce62-c0f0-4cea-a734-8749db6b0d1a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.465166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ce62-c0f0-4cea-a734-8749db6b0d1a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.465986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ce62-c0f0-4cea-a734-8749db6b0d1a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.469696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 907/1825] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:04:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:04:17.596866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 919/1826] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:17.601211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ce62-c0f0-4cea-a734-8749db6b0d1a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:17.604578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2927ce62-c0f0-4cea-a734-8749db6b0d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:17.604972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8be0d-718a-4eb8-8c43-993617e71203 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.605272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8be0d-718a-4eb8-8c43-993617e71203 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.605563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8be0d-718a-4eb8-8c43-993617e71203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.655767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-07c8be0d-718a-4eb8-8c43-993617e71203 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:17.657431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-07c8be0d-718a-4eb8-8c43-993617e71203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:17.887917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:17.903389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ce62-c0f0-4cea-a734-8749db6b0d1a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:17.915037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ca9a8-2891-4fc8-b43d-c0edd4f6c6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:17.919823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca9a8-2891-4fc8-b43d-c0edd4f6c6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.921808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca9a8-2891-4fc8-b43d-c0edd4f6c6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.922133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca9a8-2891-4fc8-b43d-c0edd4f6c6e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.924273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ca9a8-2891-4fc8-b43d-c0edd4f6c6e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:17.930380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ca9a8-2891-4fc8-b43d-c0edd4f6c6e9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:17.938436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 920/1827] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:04:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:17.977259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a86af-d1bf-426e-877c-b3e644e22c9b None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:17.977838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a86af-d1bf-426e-877c-b3e644e22c9b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:17.978628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a86af-d1bf-426e-877c-b3e644e22c9b None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:17.980417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 921/1828] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:04:17.998891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 908/1829] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:17 2022] POST /identity/v3/projects => generated 371 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:18.007171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a86af-d1bf-426e-877c-b3e644e22c9b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:18.028632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.029717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.030435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.077265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:18.080192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:18.192939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a86af-d1bf-426e-877c-b3e644e22c9b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:18.201579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:18.202565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.203276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.203997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.205557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:18.208215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:18.236336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 922/1830] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:18 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:18.241155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:18.259302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ec1d1-9602-4d85-812c-ad5526e4f570 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:18.268021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:18.269828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.270620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.271152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.272436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:18.275346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:18.287669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 909/1831] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:18.290884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 923/1832] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:18.293704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:18.306356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6bd32-32e7-4041-8b52-2657ed7d2168 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:18.311751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:18.312670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.312979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.313243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] PATH_INFO: `/v3/projects/e5cff105a1254084b082457fa7ca3cf6/users/136054ae6b7f4e788c5e11d13e246ef3/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.333666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cff105a1254084b082457fa7ca3cf6, user_id=136054ae6b7f4e788c5e11d13e246ef3, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:18.337224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:18.356875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 910/1833] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:18 2022] PUT /identity/v3/projects/e5cff105a1254084b082457fa7ca3cf6/users/136054ae6b7f4e788c5e11d13e246ef3/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:18.357956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:18.363697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:18.372826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fcd6e-a125-401b-a842-90946bf65de0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:18.467027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:18.467899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.468189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.468471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.469206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:18.471233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:18.484521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 924/1834] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:18.488455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:18.503478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3d75b-01e8-4322-a109-9ee8c30e57c9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:18.511618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e97bb-0baa-4519-b202-0bae344e26f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:18.513479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e97bb-0baa-4519-b202-0bae344e26f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.513797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e97bb-0baa-4519-b202-0bae344e26f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.514369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e97bb-0baa-4519-b202-0bae344e26f6 admin admin] PATH_INFO: `/v3/projects/e5cff105a1254084b082457fa7ca3cf6/users/136054ae6b7f4e788c5e11d13e246ef3/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.520284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e97bb-0baa-4519-b202-0bae344e26f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cff105a1254084b082457fa7ca3cf6, user_id=136054ae6b7f4e788c5e11d13e246ef3, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:18.527648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e97bb-0baa-4519-b202-0bae344e26f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:18.547708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 911/1835] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:18 2022] PUT /identity/v3/projects/e5cff105a1254084b082457fa7ca3cf6/users/136054ae6b7f4e788c5e11d13e246ef3/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:18.708326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:18.711416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:18.835248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2c3ee-cc67-4144-a00f-841105acaa09 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:18.932161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:18.933813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.934266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.937668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:18.954045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:18.958693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:18.983904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 925/1836] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:18.985915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7345a-302a-4c3c-abf2-5885c96a1c3f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:18.986264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7345a-302a-4c3c-abf2-5885c96a1c3f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:18.986678 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7345a-302a-4c3c-abf2-5885c96a1c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:19.015580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e4b7345a-302a-4c3c-abf2-5885c96a1c3f None None] MFA Rules not processed for user `c8bda0c079054e198ffeca4c0ac18bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:19.019532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e4b7345a-302a-4c3c-abf2-5885c96a1c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:19.056483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:19.074540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf063b1d-4c67-4d55-8f55-5f223c770ee1 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:19.088246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:19.089556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:19.090206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:19.090637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:19.108208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:19.111447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:19.143066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 926/1837] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:19.211869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 912/1838] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:19.739059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:19.740669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:19.762626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:19.846605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaec67-c7d7-4a35-ad5f-8476c82b8691 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:19.854535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:19.855955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:19.856350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:19.856732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] PATH_INFO: `/v3/projects/17bde3f179d9453c9f2efd4367db3a42` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:19.859270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bde3f179d9453c9f2efd4367db3a42)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:19.862134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:19.939859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bde3f179d9453c9f2efd4367db3a42', 'request_id': 'req-92381725-1892-4194-a956-820f7bd27d59'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:19.949675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bde3f179d9453c9f2efd4367db3a42', 'request_id': 'req-92381725-1892-4194-a956-820f7bd27d59'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:19.967532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bde3f179d9453c9f2efd4367db3a42', 'request_id': 'req-92381725-1892-4194-a956-820f7bd27d59'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:19.985632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] The token cache is being invalidate because project 17bde3f179d9453c9f2efd4367db3a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:19.986233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:19.987107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92381725-1892-4194-a956-820f7bd27d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:19.992693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 927/1839] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:19 2022] DELETE /identity/v3/projects/17bde3f179d9453c9f2efd4367db3a42 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:19.995216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:19.995711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:19.995798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.022980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:20.025154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:20.080460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:20.114453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 913/1840] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:20.116708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.117150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.117573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.137935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:20.139201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:20.147249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 928/1841] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:20.152628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.162391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821137ad-14a8-4328-bba0-e9f99f783c26 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.168088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.168697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.168980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.169256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.169976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:20.170284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.171920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.184557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 914/1842] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:20.188937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.198977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90323e6f-8be7-4aa5-972a-04e26c5a736d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.216362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.217137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.217433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.217934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.218405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.220553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.253939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 929/1843] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:20.256055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.271271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b636920-e08d-41da-8140-091167c5a447 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.360860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.362317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.362715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.363089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.363913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.367887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.407176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 915/1844] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:20.412721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.426483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5886d-82fe-47a5-88f3-922b24f1c461 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.432104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.432812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.433133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.433427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.434305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.436129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.463326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 930/1845] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:20.467344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.483838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e86c8-fa37-4724-9243-6aa7d9b9b3fe admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.491536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e01df-9a32-485b-8735-8132999ca514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.492999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e01df-9a32-485b-8735-8132999ca514 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.493322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e01df-9a32-485b-8735-8132999ca514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.493606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e01df-9a32-485b-8735-8132999ca514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.494470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e01df-9a32-485b-8735-8132999ca514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.496356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e01df-9a32-485b-8735-8132999ca514 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.510967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 916/1846] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:20.514735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.529393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9f9e-57b2-46e7-b663-d2acae3c9904 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.537903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479de9fb-ffc3-4250-8423-715be66f2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.539410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479de9fb-ffc3-4250-8423-715be66f2714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.540476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479de9fb-ffc3-4250-8423-715be66f2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.540984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479de9fb-ffc3-4250-8423-715be66f2714 admin admin] PATH_INFO: `/v3/projects/fefc1ada6b2c4d8b928d804a4e6b9778/users/75781314f9e843cc8c200a758e6a1efa/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.579741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479de9fb-ffc3-4250-8423-715be66f2714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fefc1ada6b2c4d8b928d804a4e6b9778, user_id=75781314f9e843cc8c200a758e6a1efa, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.582641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479de9fb-ffc3-4250-8423-715be66f2714 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.613366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 931/1847] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:20 2022] PUT /identity/v3/projects/fefc1ada6b2c4d8b928d804a4e6b9778/users/75781314f9e843cc8c200a758e6a1efa/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:20.617480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.621320 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.621823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.670035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] MFA Rules not processed for user `75781314f9e843cc8c200a758e6a1efa`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:20.671617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:20.709189 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70747810-4b42-402f-b563-14013d48783e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.709500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70747810-4b42-402f-b563-14013d48783e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.709789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70747810-4b42-402f-b563-14013d48783e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.759964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-70747810-4b42-402f-b563-14013d48783e None None] MFA Rules not processed for user `136054ae6b7f4e788c5e11d13e246ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:20.761191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-70747810-4b42-402f-b563-14013d48783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:20.783768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 917/1848] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:20.784917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.805297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:20.840200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:20.911124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a5dde-d50b-4dc7-b1c5-fb77e8766038 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:20.937260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:20.937995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:20.938563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:20.938928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:20.939630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:20.942200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:20.972549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 932/1849] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:20.978239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70747810-4b42-402f-b563-14013d48783e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:20.980094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-70747810-4b42-402f-b563-14013d48783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:20.987994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 918/1850] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/projects => generated 387 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:20.992011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:21.094784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70747810-4b42-402f-b563-14013d48783e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:21.130269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84548c51-de0f-4c9c-886f-43c24ebb7ee4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:21.139591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:21.140541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.144113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.144691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.146450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:21.149121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:21.172999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:21.173681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.173973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.174254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.176175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:21.189762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 919/1851] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:20 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:21.193487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:21.210807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86287441-5fba-44c1-8b2e-a427adedd2ff admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:21.219444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:21.223261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.223722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.228600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.229729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:21.231527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:21.246052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 920/1852] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:21.251635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:21.266814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c61f6-350a-4eb2-82bb-67ad83a45404 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:21.268125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:21.269787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:21.273766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:21.275473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.275760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.276036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] PATH_INFO: `/v3/projects/4e88fe7f3b214cd1a52d480aeb5f7756/users/4e27a2e0d0a14bea8014f5c914c64f44/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.281864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 933/1853] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:21.294344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e88fe7f3b214cd1a52d480aeb5f7756, user_id=4e27a2e0d0a14bea8014f5c914c64f44, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:21.298218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:21.330653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 921/1854] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:21 2022] PUT /identity/v3/projects/4e88fe7f3b214cd1a52d480aeb5f7756/users/4e27a2e0d0a14bea8014f5c914c64f44/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:21.331808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:21.352199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765cb16-eb39-446b-a625-5838a539c0ff service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:21.418540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:21.481301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a93f42-81a2-46eb-9379-743edab5e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:21.483445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a93f42-81a2-46eb-9379-743edab5e307 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.483734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a93f42-81a2-46eb-9379-743edab5e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.486260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a93f42-81a2-46eb-9379-743edab5e307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.487815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a93f42-81a2-46eb-9379-743edab5e307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:21.492472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a93f42-81a2-46eb-9379-743edab5e307 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:21.518645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 934/1855] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:21.520835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:21.538823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29e0d6-735b-4c1c-9d71-36a6ccf32055 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:21.564780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:21.567382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.569128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.572883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] PATH_INFO: `/v3/projects/4e88fe7f3b214cd1a52d480aeb5f7756/users/4e27a2e0d0a14bea8014f5c914c64f44/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.579332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e88fe7f3b214cd1a52d480aeb5f7756, user_id=4e27a2e0d0a14bea8014f5c914c64f44, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:21.584108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:21.603291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 922/1856] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:21 2022] PUT /identity/v3/projects/4e88fe7f3b214cd1a52d480aeb5f7756/users/4e27a2e0d0a14bea8014f5c914c64f44/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:21.613687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:21.613928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:21.614217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:21.684930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] MFA Rules not processed for user `4e27a2e0d0a14bea8014f5c914c64f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:21.686251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:21.894738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 935/1857] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:21.896719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:22.037384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3c185-cae8-48e9-939f-b38a0dea78f0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:22.044095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:22.045032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.045562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.046038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.059802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:22.062638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:22.080015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 923/1858] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:22.210262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:22.354287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a0c08-56df-49a4-9783-91a22123d5ff None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:22.365815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:22.367062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.367360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.367649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] PATH_INFO: `/v3/users/75781314f9e843cc8c200a758e6a1efa` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.371415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75781314f9e843cc8c200a758e6a1efa)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:22.375649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:22.414006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:22.418090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:22.436948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79851-413e-4182-8385-2ae97ee91c2b service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:22.444194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:22.445403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.445871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.446682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] PATH_INFO: `/v3/users/a213b8a65ff9453aa7feb15db373fbf2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.449412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a213b8a65ff9453aa7feb15db373fbf2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:22.451920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:22.453929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:22.488973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75781314f9e843cc8c200a758e6a1efa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.490745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75781314f9e843cc8c200a758e6a1efa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.510530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:22.514304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75781314f9e843cc8c200a758e6a1efa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.515386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75781314f9e843cc8c200a758e6a1efa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.532839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fa771679-b3b9-44f6-9a0d-fd1b474128b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75781314f9e843cc8c200a758e6a1efa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.534760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a213b8a65ff9453aa7feb15db373fbf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.538661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a213b8a65ff9453aa7feb15db373fbf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.539569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a213b8a65ff9453aa7feb15db373fbf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.558718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 936/1859] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:22 2022] DELETE /identity/v3/users/75781314f9e843cc8c200a758e6a1efa => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:22.563083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.563547 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.564797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.575811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a213b8a65ff9453aa7feb15db373fbf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.594545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:22.596959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:22.626918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3867fff-87ba-4442-85e4-c4fae123291e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a213b8a65ff9453aa7feb15db373fbf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:22.654229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 924/1860] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:22 2022] DELETE /identity/v3/users/a213b8a65ff9453aa7feb15db373fbf2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:22.659255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.659550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.659831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.695524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 937/1861] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:22.709367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:22.710707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:22.733162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:22.734818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:22.847998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 925/1862] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:22.853018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299b35c-ee75-4646-aca4-23989f6d5331 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:22.860048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:22.922945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:22.923545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.923829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.924110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.929964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a582aa-65d2-45ea-9897-1c92f63030b4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:22.934496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:22.934716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:22.935350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:22.935741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:22.936030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:22.936155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:22.947875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:22.949521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:22.951013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 938/1863] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:22.960724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 926/1864] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:23.033997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:23.035950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:23.141555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:23.224018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939d3b-6343-4960-ab67-6b63283ef628 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:23.238900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:23.239888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:23.240275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:23.240657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:23.242865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:23.360263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:23.361945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:23.382382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 939/1865] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:23.954010 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:23.956613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:23.980955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80638380-71b3-465b-8d86-5eecf01556e3 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:23.990944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:23.991840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:23.992131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:23.992567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] PATH_INFO: `/v3/projects/fc3006407e034228af618bcf92776f07` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:23.994675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3006407e034228af618bcf92776f07)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:23.996849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:24.046383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:24.072378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:24.073874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:24.107998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5d3d3-1c05-4cf0-8720-ea8f8c05d26f service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:24.118193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3006407e034228af618bcf92776f07', 'request_id': 'req-7c964769-80a4-4b06-899d-80d92e71a056'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.135465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3006407e034228af618bcf92776f07', 'request_id': 'req-7c964769-80a4-4b06-899d-80d92e71a056'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.150985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3006407e034228af618bcf92776f07', 'request_id': 'req-7c964769-80a4-4b06-899d-80d92e71a056'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.176439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] The token cache is being invalidate because project fc3006407e034228af618bcf92776f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:24.177068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.177761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.179847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 927/1866] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:23 2022] DELETE /identity/v3/projects/fc3006407e034228af618bcf92776f07 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:24.184807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:24.186380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:24.252105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:24.253045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:24.253340 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:24.253626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] PATH_INFO: `/v3/projects/fefc1ada6b2c4d8b928d804a4e6b9778` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:24.255294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fefc1ada6b2c4d8b928d804a4e6b9778)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:24.258147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:24.282472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c964769-80a4-4b06-899d-80d92e71a056 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:24.288600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:24.290720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:24.291051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:24.291870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] PATH_INFO: `/v3/users/5c5daf6fb5794fc9a3b8471b39cc79cd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:24.293695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5daf6fb5794fc9a3b8471b39cc79cd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:24.296444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:24.330923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fefc1ada6b2c4d8b928d804a4e6b9778', 'request_id': 'req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.337057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fefc1ada6b2c4d8b928d804a4e6b9778', 'request_id': 'req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.350287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fefc1ada6b2c4d8b928d804a4e6b9778', 'request_id': 'req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.363914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] The token cache is being invalidate because project fefc1ada6b2c4d8b928d804a4e6b9778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:24.364544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.365047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.366879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 940/1867] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:24 2022] DELETE /identity/v3/projects/fefc1ada6b2c4d8b928d804a4e6b9778 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:24.367449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5daf6fb5794fc9a3b8471b39cc79cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.368236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5daf6fb5794fc9a3b8471b39cc79cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.368887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5daf6fb5794fc9a3b8471b39cc79cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.370532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:24.372151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:24.380250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5daf6fb5794fc9a3b8471b39cc79cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.391484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:24.399873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5daf6fb5794fc9a3b8471b39cc79cd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.425956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 928/1868] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:24 2022] DELETE /identity/v3/users/5c5daf6fb5794fc9a3b8471b39cc79cd => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:24.470104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db2a1-6c28-4d65-b7b6-9f88688bcf3e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:24.541146 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:24.542251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:24.542614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:24.543000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] PATH_INFO: `/v3/users/4e27a2e0d0a14bea8014f5c914c64f44` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:24.544268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e27a2e0d0a14bea8014f5c914c64f44)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:24.546261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:24.622127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e27a2e0d0a14bea8014f5c914c64f44'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.622809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e27a2e0d0a14bea8014f5c914c64f44'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.631115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e27a2e0d0a14bea8014f5c914c64f44'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.631842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e27a2e0d0a14bea8014f5c914c64f44'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.640192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-38291589-dc68-4a86-9513-ed998e413b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e27a2e0d0a14bea8014f5c914c64f44'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.675162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 941/1869] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:24 2022] DELETE /identity/v3/users/4e27a2e0d0a14bea8014f5c914c64f44 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:24.805053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:24.807152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:24.824086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bec055-1ca9-46ec-a8f8-a6ceebfdf03a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:24.857468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:24.910194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:24.911467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:24.912340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:24.912768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] PATH_INFO: `/v3/projects/c4cdb966224b4f0f82ceab9047241199` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:24.926163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4cdb966224b4f0f82ceab9047241199)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:24.931967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:24.992416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cdb966224b4f0f82ceab9047241199', 'request_id': 'req-cc4352b2-3569-4f31-b389-f5538d9b6c7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:24.999685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cdb966224b4f0f82ceab9047241199', 'request_id': 'req-cc4352b2-3569-4f31-b389-f5538d9b6c7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.011349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cdb966224b4f0f82ceab9047241199', 'request_id': 'req-cc4352b2-3569-4f31-b389-f5538d9b6c7e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.027740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] The token cache is being invalidate because project c4cdb966224b4f0f82ceab9047241199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:25.028061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.028770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.030619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 929/1870] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:24 2022] DELETE /identity/v3/projects/c4cdb966224b4f0f82ceab9047241199 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:25.036699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742ee48-57cd-4364-82b8-c7f042961503 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.037108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742ee48-57cd-4364-82b8-c7f042961503 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.037493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742ee48-57cd-4364-82b8-c7f042961503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.061556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2742ee48-57cd-4364-82b8-c7f042961503 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:25.062917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2742ee48-57cd-4364-82b8-c7f042961503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.112316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:25.147155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.149630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.173249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 942/1871] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:25.182022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.183015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.183117 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.208145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:25.209629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.223842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 943/1872] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:25.228644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.239366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5966d-62ae-4a94-8e37-23203cab540d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.245525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.246546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.247157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.247461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.248366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:25.248811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.251146 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.261257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4352b2-3569-4f31-b389-f5538d9b6c7e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.265154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 944/1873] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:04:25.268707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.269319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.269890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.270152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.270429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] PATH_INFO: `/v3/projects/4e88fe7f3b214cd1a52d480aeb5f7756` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.271669 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e88fe7f3b214cd1a52d480aeb5f7756)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.273589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.281406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68398b46-85fb-4bfb-b4f9-715d6d6f8d5f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.291090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.291871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.292415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.293268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.294384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.298264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.338952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e88fe7f3b214cd1a52d480aeb5f7756', 'request_id': 'req-ef7ea260-3e26-4913-853d-45dc6cad3dbe'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.342226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 945/1874] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:25.343331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.347683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:25.349858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e88fe7f3b214cd1a52d480aeb5f7756', 'request_id': 'req-ef7ea260-3e26-4913-853d-45dc6cad3dbe'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.363491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e88fe7f3b214cd1a52d480aeb5f7756', 'request_id': 'req-ef7ea260-3e26-4913-853d-45dc6cad3dbe'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.384893 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] The token cache is being invalidate because project 4e88fe7f3b214cd1a52d480aeb5f7756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:25.385833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.386308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef7ea260-3e26-4913-853d-45dc6cad3dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:25.392259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 930/1875] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:25 2022] DELETE /identity/v3/projects/4e88fe7f3b214cd1a52d480aeb5f7756 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:25.394055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9639b-3a10-41fc-8a25-09532dcc46e8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.394472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9639b-3a10-41fc-8a25-09532dcc46e8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.394883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9639b-3a10-41fc-8a25-09532dcc46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.416361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed2509-283b-4a45-a5a8-8c8fa53eadb9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.420420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-93c9639b-3a10-41fc-8a25-09532dcc46e8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:25.422985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93c9639b-3a10-41fc-8a25-09532dcc46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.429653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.430354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.431258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.431584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.432282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.433897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 931/1876] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:25.442803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.443140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.443427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.444471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.465806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:25.467860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.480864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 932/1877] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:25.487356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 946/1878] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:25.487930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.488680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.490170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.506312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8653064-d585-4d06-af5d-3cf1a90df372 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.512026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bead9c2-8df3-4d47-a0e5-679d74bfec9e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.512418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.514118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.514118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.514118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.514897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:25.515244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.516865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.518906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.520127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.520127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.520331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.521255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.522968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.540751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 947/1879] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:25.541194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 933/1880] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:25.544360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.548199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.560398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed823fa4-82f2-4983-9a68-584a184037ab admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.560643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2fa50-3e6c-4571-8444-f3d60edb4b1c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.567169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.567922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.568101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.568661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.568777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.568992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.569209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.569315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.570018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.570349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.573873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.574760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.591463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 934/1881] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:25.592897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.605277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f865-e567-4547-b6a7-151f27188fb7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.614005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd0cba-8837-40da-b37e-f648d3a70d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.616100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd0cba-8837-40da-b37e-f648d3a70d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.616767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd0cba-8837-40da-b37e-f648d3a70d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.617400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd0cba-8837-40da-b37e-f648d3a70d72 admin admin] PATH_INFO: `/v3/projects/d5f8a773bdfd436d8f44c8111854ba23/users/f44e355432aa4e58a928e8518a92407c/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.620047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 948/1882] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:25.623805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.642562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd0cba-8837-40da-b37e-f648d3a70d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f8a773bdfd436d8f44c8111854ba23, user_id=f44e355432aa4e58a928e8518a92407c, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.646668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd0cba-8837-40da-b37e-f648d3a70d72 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.663709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 935/1883] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:25 2022] PUT /identity/v3/projects/d5f8a773bdfd436d8f44c8111854ba23/users/f44e355432aa4e58a928e8518a92407c/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:25.674224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.674525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.674808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.703858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] MFA Rules not processed for user `f44e355432aa4e58a928e8518a92407c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:25.705079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:25.705372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:25.720409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f5ad9-220d-4d18-bc74-4e0025ea3855 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.781703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 936/1884] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:25.787915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.788092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.788595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.788879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.789156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.790386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.792418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.799473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fb24a-fb08-485b-8dfd-35b0f981b68a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.804111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.804756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.805065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.805351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.806662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.808611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.815714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 949/1885] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:25.819813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.831032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6e08-f7f5-48ce-88a5-a71d63e8ac15 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.836409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.837480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.837911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.838186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.838952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.840837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.846907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 937/1886] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:25.850555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.853199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 950/1887] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:25.857162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.867007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b687b-0168-473c-86a5-5e637ff1feda admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.926635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:25.944452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.945864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.946823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.947716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.948498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.950661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba42b-3272-4203-bba3-824f1becafa4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.951338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.963860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:25.968993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 951/1888] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:25.969162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:25.969162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:25.969162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:25.969162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:25.969752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:25.970775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:25.991576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fee01e-2633-421d-81f8-ef8efdcbc4c3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:25.999721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 938/1889] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:25 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:26.005278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8168390-0106-40db-8cc4-cea5d87294e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.006261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8168390-0106-40db-8cc4-cea5d87294e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.006725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8168390-0106-40db-8cc4-cea5d87294e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.007126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.007732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8168390-0106-40db-8cc4-cea5d87294e4 admin admin] PATH_INFO: `/v3/projects/a4f37ea6a3c14400953deba9e45bedd3/users/1d8ea794bc0b44cab523e440b6b35fda/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.022302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bca9d-9a88-4058-9f2f-725e80218aee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.027693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.028362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.028648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.028930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.029632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8168390-0106-40db-8cc4-cea5d87294e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f37ea6a3c14400953deba9e45bedd3, user_id=1d8ea794bc0b44cab523e440b6b35fda, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.029736 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.031321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.032439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8168390-0106-40db-8cc4-cea5d87294e4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.045059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 939/1890] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:26.050529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.054510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 952/1891] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:25 2022] PUT /identity/v3/projects/a4f37ea6a3c14400953deba9e45bedd3/users/1d8ea794bc0b44cab523e440b6b35fda/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:26.061282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d0325-17fb-4ec1-9a72-d38dd8010f95 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.065467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266df198-e539-4bdb-a901-afa354da9907 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.065769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266df198-e539-4bdb-a901-afa354da9907 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.066052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266df198-e539-4bdb-a901-afa354da9907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.093269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-266df198-e539-4bdb-a901-afa354da9907 None None] MFA Rules not processed for user `1d8ea794bc0b44cab523e440b6b35fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:26.094498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-266df198-e539-4bdb-a901-afa354da9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:26.119492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.120160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.120457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.120741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.121459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.123197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.132967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:26.133527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 940/1892] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:26.139495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.149810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d097749-18e0-4f30-877f-3460920af157 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.155004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b34a66-a09e-46fa-889b-7e6a5a09cf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.155952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b34a66-a09e-46fa-889b-7e6a5a09cf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.156196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b34a66-a09e-46fa-889b-7e6a5a09cf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.156725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b34a66-a09e-46fa-889b-7e6a5a09cf8f admin admin] PATH_INFO: `/v3/projects/fb241841e1b14b0fa7fb2403a34822a6/users/fd9ec1dc0c6f4a5db320cfc60e037f63/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.166251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 953/1893] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:26.170812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266df198-e539-4bdb-a901-afa354da9907 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.172304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b34a66-a09e-46fa-889b-7e6a5a09cf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb241841e1b14b0fa7fb2403a34822a6, user_id=fd9ec1dc0c6f4a5db320cfc60e037f63, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.175013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b34a66-a09e-46fa-889b-7e6a5a09cf8f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.184176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266df198-e539-4bdb-a901-afa354da9907 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.192043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 941/1894] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:26 2022] PUT /identity/v3/projects/fb241841e1b14b0fa7fb2403a34822a6/users/fd9ec1dc0c6f4a5db320cfc60e037f63/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:26.204302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.204607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.205204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.234540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] MFA Rules not processed for user `fd9ec1dc0c6f4a5db320cfc60e037f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:26.235880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:26.267112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.268072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.268263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.268616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.269426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.271889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.311027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 954/1895] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:26.313115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.334062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 942/1896] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:26.339452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.351138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b4236-0128-47a8-a915-2e7b7c3049bf None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.390063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:26.398230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1263-4593-4e3a-a879-9e3785cb9b44 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.404179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.404934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.405294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.405578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.406399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.408377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.425132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.425806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.426094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.426376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.427145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.429351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.438972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 955/1897] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:26.442027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.459075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b6f2c-66e8-4b19-b639-c6caaeb9defe admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.471570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.472721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.473194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.473636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.474610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.476430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.487730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 943/1898] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:26.488837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.489988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 956/1899] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:26.494333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.512973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7290d-6a6d-41d5-859c-a6f4138a0565 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.618232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:26.626562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ae02-9089-4419-9a5a-eb1c3256b53f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.631939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.632683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.632969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.633250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.634702 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.636974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.642484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.643568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.643934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.644310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] PATH_INFO: `/v3/projects/c77452e248c049cfa4329bcf7a50fb8f/users/34362a1857b9455baef420910cbc978f/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.675936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77452e248c049cfa4329bcf7a50fb8f, user_id=34362a1857b9455baef420910cbc978f, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.679692 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.682115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 944/1900] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:26.686657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.710886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 957/1901] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:26 2022] PUT /identity/v3/projects/c77452e248c049cfa4329bcf7a50fb8f/users/34362a1857b9455baef420910cbc978f/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:26.713577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f8ee-587c-4926-a656-0b60a8062048 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.715617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.823348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.825445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.826141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.826655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.827778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.830691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.850405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a0cb1-c6bf-4f47-ae5d-024ca1ae4e36 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.853058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 945/1902] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:26.857751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.860735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.862484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.862973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.863357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.864982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.868853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.872732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f97174-fcf6-4e53-8275-3e482b6cda2f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.879373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.880890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.881494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.881826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] PATH_INFO: `/v3/projects/323a3a857a054a22b6668ccb2c2393e4/users/805746ae69024c1aa30d7fc6d6f50520/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.893612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 958/1903] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:26.898026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.907349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=323a3a857a054a22b6668ccb2c2393e4, user_id=805746ae69024c1aa30d7fc6d6f50520, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.910277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.911187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657a943-c6e2-4336-9915-4d145d99a25d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:26.917029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51a853-5db8-4fa6-b7b3-a86303c5a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:26.918607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51a853-5db8-4fa6-b7b3-a86303c5a693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.918955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51a853-5db8-4fa6-b7b3-a86303c5a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.919242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51a853-5db8-4fa6-b7b3-a86303c5a693 admin admin] PATH_INFO: `/v3/projects/c77452e248c049cfa4329bcf7a50fb8f/users/34362a1857b9455baef420910cbc978f/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:26.924400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51a853-5db8-4fa6-b7b3-a86303c5a693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77452e248c049cfa4329bcf7a50fb8f, user_id=34362a1857b9455baef420910cbc978f, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:26.927688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51a853-5db8-4fa6-b7b3-a86303c5a693 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:26.952318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 946/1905] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:26 2022] PUT /identity/v3/projects/323a3a857a054a22b6668ccb2c2393e4/users/805746ae69024c1aa30d7fc6d6f50520/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:26.956162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 959/1905] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:26 2022] PUT /identity/v3/projects/c77452e248c049cfa4329bcf7a50fb8f/users/34362a1857b9455baef420910cbc978f/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:26.960025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:26.964386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:26.964777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:26.965396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.003239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:27.020439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] MFA Rules not processed for user `34362a1857b9455baef420910cbc978f`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:27.022758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:27.072793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d885ba-2343-4e6f-bdfb-a037a8978127 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:27.133331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 960/1906] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:27.143179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:27.145265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:27.153592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:27.154768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.155252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.155733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.157220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:27.160379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:27.172724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 947/1907] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:27.176982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:27.191960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:27.193362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680ac6-4b8f-4d65-9000-733607769203 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:27.200074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d24dc-f2df-458c-99e2-59ace3390add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:27.200994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24dc-f2df-458c-99e2-59ace3390add admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.201280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24dc-f2df-458c-99e2-59ace3390add admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.201564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24dc-f2df-458c-99e2-59ace3390add admin admin] PATH_INFO: `/v3/projects/323a3a857a054a22b6668ccb2c2393e4/users/805746ae69024c1aa30d7fc6d6f50520/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.203673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d24dc-f2df-458c-99e2-59ace3390add admin admin] RBAC: Authorizing `identity:create_grant(project_id=323a3a857a054a22b6668ccb2c2393e4, user_id=805746ae69024c1aa30d7fc6d6f50520, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:27.208323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d24dc-f2df-458c-99e2-59ace3390add admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:27.229164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 948/1908] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:27 2022] PUT /identity/v3/projects/323a3a857a054a22b6668ccb2c2393e4/users/805746ae69024c1aa30d7fc6d6f50520/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:27.229371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28f79e-fc55-49b3-9fde-8a5e5282d4e0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:27.233970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.234588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.235077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.268394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] MFA Rules not processed for user `805746ae69024c1aa30d7fc6d6f50520`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:27.269755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:27.327352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:27.328027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.328441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.328729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.414001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 949/1909] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:27.430820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:27.433544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:27.453167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bef94-ef63-4361-8f28-1c44e2512736 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:27.455465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:27.458024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:27.458280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:27.458936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.459408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.459766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.473799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 961/1910] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:27.476821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:27.479176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:27.491180 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 950/1911] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:27.573225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:27.588009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cce35b-687c-4005-88a8-843feb1d14e8 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:27.599658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:27.600454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.600578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:27.600799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.601136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.610866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20846ed3-5b22-41b8-8370-7c950304d5f8 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:27.615065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2ad6-a3bb-40ac-9154-5baafff16df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:27.615791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2ad6-a3bb-40ac-9154-5baafff16df9 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:27.616011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2ad6-a3bb-40ac-9154-5baafff16df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:27.616291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2ad6-a3bb-40ac-9154-5baafff16df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:27.641297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:27.678945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:27.681001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:27.684593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe2ad6-a3bb-40ac-9154-5baafff16df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:27.686086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe2ad6-a3bb-40ac-9154-5baafff16df9 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:27.692445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 962/1912] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:27.697695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 951/1913] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:30.415655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:30.481890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:30.517612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39488e8-91ac-497b-8e34-e0a0ac122b70 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:30.522226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:30.523375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:30.523800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:30.524926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] PATH_INFO: `/v3/users/1d8ea794bc0b44cab523e440b6b35fda` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:30.527530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d8ea794bc0b44cab523e440b6b35fda)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:30.529642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:30.615337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d8ea794bc0b44cab523e440b6b35fda'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:30.617715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d8ea794bc0b44cab523e440b6b35fda'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:30.630150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d8ea794bc0b44cab523e440b6b35fda'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:30.630939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d8ea794bc0b44cab523e440b6b35fda'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:30.642733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d8ea794bc0b44cab523e440b6b35fda'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:30.668993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 963/1914] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:30 2022] DELETE /identity/v3/users/1d8ea794bc0b44cab523e440b6b35fda => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:30.673704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:30.674331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:30.674446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:30.732160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:30.733435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:30.848232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 952/1915] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:30.888298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:30.890979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:31.004210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e2135-c584-42fd-86d9-ee7f41dc7b30 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:31.014929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:31.014929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:31.014929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:31.014929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:31.027006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:31.029207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:31.053851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 964/1916] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:31.664801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:31.667332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:31.690917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250eb0-e18b-49e1-ac7c-c6b3c1ab8c63 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:31.706986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:31.708011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:31.708376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:31.708738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] PATH_INFO: `/v3/projects/a4f37ea6a3c14400953deba9e45bedd3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:31.710211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f37ea6a3c14400953deba9e45bedd3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:31.712649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:31.824018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f37ea6a3c14400953deba9e45bedd3', 'request_id': 'req-279532f9-6043-4d06-ab10-3cab91971742'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:31.835506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f37ea6a3c14400953deba9e45bedd3', 'request_id': 'req-279532f9-6043-4d06-ab10-3cab91971742'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:31.848841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f37ea6a3c14400953deba9e45bedd3', 'request_id': 'req-279532f9-6043-4d06-ab10-3cab91971742'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:31.875703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] The token cache is being invalidate because project a4f37ea6a3c14400953deba9e45bedd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:31.876126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:31.878888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:31.882083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 953/1917] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:31 2022] DELETE /identity/v3/projects/a4f37ea6a3c14400953deba9e45bedd3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:31.886112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:31.899268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505f8de-59ec-4133-b9c1-92b8992a1ad2 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:32.028630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:32.029972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:32.030132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:32.030477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] PATH_INFO: `/v3/users/34362a1857b9455baef420910cbc978f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:32.032882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34362a1857b9455baef420910cbc978f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:32.035067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:32.037132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:32.142560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34362a1857b9455baef420910cbc978f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.144221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34362a1857b9455baef420910cbc978f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.173974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34362a1857b9455baef420910cbc978f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.177478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34362a1857b9455baef420910cbc978f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.194412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-93c8a2af-83b0-44ea-b2a4-0f1e1a9e6b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34362a1857b9455baef420910cbc978f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.260116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 965/1918] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:31 2022] DELETE /identity/v3/users/34362a1857b9455baef420910cbc978f => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:32.344104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:32.345826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:32.368634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279532f9-6043-4d06-ab10-3cab91971742 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:32.439036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:32.473198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:32.474989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:32.475554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:32.476091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] PATH_INFO: `/v3/users/f44e355432aa4e58a928e8518a92407c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:32.478813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44e355432aa4e58a928e8518a92407c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:32.484375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:32.584401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44e355432aa4e58a928e8518a92407c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.585138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44e355432aa4e58a928e8518a92407c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.585822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44e355432aa4e58a928e8518a92407c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.596922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44e355432aa4e58a928e8518a92407c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.619864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44e355432aa4e58a928e8518a92407c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:32.637769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 954/1919] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:32 2022] DELETE /identity/v3/users/f44e355432aa4e58a928e8518a92407c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:32.643616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:32.643851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:32.644129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:32.662814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:32.664108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:32.756149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 966/1920] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:32.773950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:32.776640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:32.879333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d81a7-470a-40a5-a0c8-f25599e29464 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:32.884153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc5a9c-befa-430f-b931-f372b92f5c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:32.884831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5a9c-befa-430f-b931-f372b92f5c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:32.885122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5a9c-befa-430f-b931-f372b92f5c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:32.885516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5a9c-befa-430f-b931-f372b92f5c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:32.904012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5a9c-befa-430f-b931-f372b92f5c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:32.908732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5a9c-befa-430f-b931-f372b92f5c4d service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:32.923533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 955/1921] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:04:32.928295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:32.929753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:32.942444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d54a7-101c-44a2-907b-21a03266e2e7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:32.949309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:32.949954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:32.950311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:32.950664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] PATH_INFO: `/v3/projects/c77452e248c049cfa4329bcf7a50fb8f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:32.952124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77452e248c049cfa4329bcf7a50fb8f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:32.954555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.023151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77452e248c049cfa4329bcf7a50fb8f', 'request_id': 'req-29098b9f-0fb7-44ff-8f78-cd228763ddcb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.031438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77452e248c049cfa4329bcf7a50fb8f', 'request_id': 'req-29098b9f-0fb7-44ff-8f78-cd228763ddcb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.040570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77452e248c049cfa4329bcf7a50fb8f', 'request_id': 'req-29098b9f-0fb7-44ff-8f78-cd228763ddcb'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.052209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] The token cache is being invalidate because project c77452e248c049cfa4329bcf7a50fb8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:33.052627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.053397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-29098b9f-0fb7-44ff-8f78-cd228763ddcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.055142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 967/1922] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:32 2022] DELETE /identity/v3/projects/c77452e248c049cfa4329bcf7a50fb8f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:33.060565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.060860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.061147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.081229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:33.082715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.147737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 956/1923] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:33.157624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.157991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.158443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.166091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:33.183874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:33.185110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.193043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 968/1924] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:33.197803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.207975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed23e51-fc0b-43ba-92c8-e1a3b56d7fac None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.212670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.213333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.213625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.213909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.214782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:33.215137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.217298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.218919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:33.236074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 957/1925] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:04:33.237383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.248473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7de09-87d6-40ad-a4a5-1687d0d24c1a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.253713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.254424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.254802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.255174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.255881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.257839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.258588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.261781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.281130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd8ca9-760b-4ab1-9ef4-88df0b26a959 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.286664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.287734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.288104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.288448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] PATH_INFO: `/v3/projects/d5f8a773bdfd436d8f44c8111854ba23` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.289681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 969/1926] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:33 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:33.289886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f8a773bdfd436d8f44c8111854ba23)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.292160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.294499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.356141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f8a773bdfd436d8f44c8111854ba23', 'request_id': 'req-bb12e6e4-3487-46de-b737-f310ba3b2465'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.362654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f8a773bdfd436d8f44c8111854ba23', 'request_id': 'req-bb12e6e4-3487-46de-b737-f310ba3b2465'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.370073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f8a773bdfd436d8f44c8111854ba23', 'request_id': 'req-bb12e6e4-3487-46de-b737-f310ba3b2465'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.381279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45ac93-e849-4499-89b4-e764c6821147 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.389645 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] The token cache is being invalidate because project d5f8a773bdfd436d8f44c8111854ba23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:33.390247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.391525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.394127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 958/1927] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:33 2022] DELETE /identity/v3/projects/d5f8a773bdfd436d8f44c8111854ba23 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:33.398154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.399674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.487825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e6e4-3487-46de-b737-f310ba3b2465 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.497169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.498698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.499204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.499737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] PATH_INFO: `/v3/users/fd9ec1dc0c6f4a5db320cfc60e037f63` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.501768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9ec1dc0c6f4a5db320cfc60e037f63)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.504608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.520126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.520950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.521637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.521637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.522534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.525785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.563107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 970/1928] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:33 2022] POST /identity/v3/users => generated 287 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:33.567826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.572201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.578623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9ec1dc0c6f4a5db320cfc60e037f63'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.579643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9ec1dc0c6f4a5db320cfc60e037f63'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.580461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9ec1dc0c6f4a5db320cfc60e037f63'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.592164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9ec1dc0c6f4a5db320cfc60e037f63'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.593256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b34e15-6de6-4c3a-ba1f-309b1de8de07 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.600758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.601795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.602529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.603224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.604807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.609530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.629368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 971/1929] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:33.633841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.636809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.641366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9ec1dc0c6f4a5db320cfc60e037f63'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:33.653848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:33.656563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25295cc-f5e5-4658-bbbb-40965b876064 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.662744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f743-37e0-4d1c-9339-926a98dd9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.663540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f743-37e0-4d1c-9339-926a98dd9da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.664120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f743-37e0-4d1c-9339-926a98dd9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.664416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f743-37e0-4d1c-9339-926a98dd9da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.665798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f743-37e0-4d1c-9339-926a98dd9da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.668456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f743-37e0-4d1c-9339-926a98dd9da2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.686946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 959/1930] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:33 2022] DELETE /identity/v3/users/fd9ec1dc0c6f4a5db320cfc60e037f63 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:33.690940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 972/1931] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:33.695648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:33.782770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cbe94-2c3d-48cf-98d6-07ceda39199d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:33.787847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:33.788805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.789094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.789378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] PATH_INFO: `/v3/projects/0eb6ad7579554793b653c93a71fa9ff4/users/b33b5828648040a5a92477af9ae49138/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.807985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb6ad7579554793b653c93a71fa9ff4, user_id=b33b5828648040a5a92477af9ae49138, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:33.810784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:33.834910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 960/1932] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:33 2022] PUT /identity/v3/projects/0eb6ad7579554793b653c93a71fa9ff4/users/b33b5828648040a5a92477af9ae49138/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:33.840650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:33.841808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:33.841808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:33.878943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] MFA Rules not processed for user `b33b5828648040a5a92477af9ae49138`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:33.880275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:33.925660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:33.965890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 973/1933] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:33.967629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.049499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929bdb1-b8ca-4a6d-8d87-2aee3f57a416 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.054775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.055535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.056002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.056354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.057294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.059408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.075299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.076730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.094512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 961/1934] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:33 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:34.096020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.138439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54f51b-eeeb-4140-ad18-3cf18f90e807 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.173207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ab16-62ec-47a8-a2d4-b0c63d14f92a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.178043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.178828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.179265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.179637 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.180418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.182683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.191792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:34.207600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 962/1935] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:34 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:34.211397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.217373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.218409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.218808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.219200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] PATH_INFO: `/v3/projects/fb241841e1b14b0fa7fb2403a34822a6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.220482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb241841e1b14b0fa7fb2403a34822a6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.222235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ece3e-256a-4880-b379-d814bc2e77d5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.222573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.230209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.230954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.231279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.231519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.232368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.234555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.246888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 963/1936] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:34.250802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.262391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf270c5-fb73-4100-a77d-3f901c794866 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.277836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb241841e1b14b0fa7fb2403a34822a6', 'request_id': 'req-f127e8c6-dbe6-404f-a16d-ad2873ff4998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.285333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb241841e1b14b0fa7fb2403a34822a6', 'request_id': 'req-f127e8c6-dbe6-404f-a16d-ad2873ff4998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.293451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb241841e1b14b0fa7fb2403a34822a6', 'request_id': 'req-f127e8c6-dbe6-404f-a16d-ad2873ff4998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.306874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] The token cache is being invalidate because project fb241841e1b14b0fa7fb2403a34822a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:34.307169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.307844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.309467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 974/1937] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:34 2022] DELETE /identity/v3/projects/fb241841e1b14b0fa7fb2403a34822a6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:34.313318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.314932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.323260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.324166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.324460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.324746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] PATH_INFO: `/v3/projects/1dc929b713a94027b5584e89484ef548/users/4af9334154c14442b47d3e9f64bf02b2/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.340206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc929b713a94027b5584e89484ef548, user_id=4af9334154c14442b47d3e9f64bf02b2, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.343247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.357434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 964/1938] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:34 2022] PUT /identity/v3/projects/1dc929b713a94027b5584e89484ef548/users/4af9334154c14442b47d3e9f64bf02b2/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:34.361352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.362969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.384916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127e8c6-dbe6-404f-a16d-ad2873ff4998 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.401825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:34.454176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e808056-8b2b-4de9-b1b6-74f133e02221 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.458855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.459558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.459679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.459845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.460127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.460574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.460858 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.460957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.461137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] PATH_INFO: `/v3/users/805746ae69024c1aa30d7fc6d6f50520` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.462928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805746ae69024c1aa30d7fc6d6f50520)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.463114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.465166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.478520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 965/1939] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:34.479954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.489695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d7d4-ac04-4432-a2a2-1ea70c1d6fd7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.495169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2971-fa68-4c7b-985e-597603655752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.496089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2971-fa68-4c7b-985e-597603655752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.496373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2971-fa68-4c7b-985e-597603655752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.496648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2971-fa68-4c7b-985e-597603655752 admin admin] PATH_INFO: `/v3/projects/1dc929b713a94027b5584e89484ef548/users/4af9334154c14442b47d3e9f64bf02b2/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.498616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2971-fa68-4c7b-985e-597603655752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc929b713a94027b5584e89484ef548, user_id=4af9334154c14442b47d3e9f64bf02b2, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.501550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2971-fa68-4c7b-985e-597603655752 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.512393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 966/1940] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:34 2022] PUT /identity/v3/projects/1dc929b713a94027b5584e89484ef548/users/4af9334154c14442b47d3e9f64bf02b2/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:34.515994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805746ae69024c1aa30d7fc6d6f50520'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.516682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805746ae69024c1aa30d7fc6d6f50520'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.522929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.523215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.523496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.524032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805746ae69024c1aa30d7fc6d6f50520'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.525281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805746ae69024c1aa30d7fc6d6f50520'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.533055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805746ae69024c1aa30d7fc6d6f50520'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:34.550908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] MFA Rules not processed for user `4af9334154c14442b47d3e9f64bf02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:34.552110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.553270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 975/1941] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:34 2022] DELETE /identity/v3/users/805746ae69024c1aa30d7fc6d6f50520 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:34.594812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:34.640888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 967/1942] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:34.652933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.654370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.722540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1603cb-eec4-4c63-b60e-ead5774c8991 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.728998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.730292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.730583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.731239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.733588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.803155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:34.826270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.828442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953db39d-f948-41e2-ab18-f3940771cbfb service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:34.840722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 976/1943] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:34.876043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:34.879964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:34.984346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4fd91-4e75-40c1-b8da-d13509fe27a0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:34.989365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:34.990258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:34.990548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:34.990832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] PATH_INFO: `/v3/projects/323a3a857a054a22b6668ccb2c2393e4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:34.992084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=323a3a857a054a22b6668ccb2c2393e4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:34.995426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.071150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '323a3a857a054a22b6668ccb2c2393e4', 'request_id': 'req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:35.089594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '323a3a857a054a22b6668ccb2c2393e4', 'request_id': 'req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:35.100314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '323a3a857a054a22b6668ccb2c2393e4', 'request_id': 'req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:35.117786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] The token cache is being invalidate because project 323a3a857a054a22b6668ccb2c2393e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:35.118748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:35.120058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-66c6778f-2b33-49e6-a71b-15c1ae35d7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:35.128392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 968/1944] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:34 2022] DELETE /identity/v3/projects/323a3a857a054a22b6668ccb2c2393e4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:35.129695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.130169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.130473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.166795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:35.169231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:35.341483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 977/1945] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:35.350888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.351205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.351476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.374140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:35.375423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:35.392189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 969/1946] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:35.396279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.410065 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6d1f3-2656-44f2-a460-1ae5f34ec1d1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.416068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.417459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.417459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.417459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.418304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:35.418792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.420504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.436756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 978/1947] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:35.441215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.451523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da182018-0fed-4f7c-823b-3db75c8e4add None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.456545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.457302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.457617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.457901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.459248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.460922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.499092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 970/1948] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:35 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:35.503307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.514429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed235bc-2f09-4450-be9c-13de4bd03d7b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.582721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.583521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.583894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.585841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.585841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.586825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.612633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 979/1949] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:35 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:35.616803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.630789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d467c-c000-490a-9b1a-7ed016bdc0b7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.640639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.641429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.641809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.642190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.643026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.644840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.662328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 971/1950] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:35.664028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.677027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4e0be-0092-4b90-bb78-0d65eb423cd5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.683437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34c85d-3772-4dad-a21a-23b0ec5270fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.684187 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34c85d-3772-4dad-a21a-23b0ec5270fa admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.684482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34c85d-3772-4dad-a21a-23b0ec5270fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.684974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34c85d-3772-4dad-a21a-23b0ec5270fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.685740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34c85d-3772-4dad-a21a-23b0ec5270fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.687493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34c85d-3772-4dad-a21a-23b0ec5270fa admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.700030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 980/1951] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:35.703985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.714295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9f57b-f305-4102-bf1b-0f5ea5aca342 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.720465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.721426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.721652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.721929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] PATH_INFO: `/v3/projects/2cfd66dfb54248258c66e23eda6084c9/users/bc8d5aec013b4ca894ff14c834d9a6a2/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.739496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfd66dfb54248258c66e23eda6084c9, user_id=bc8d5aec013b4ca894ff14c834d9a6a2, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.742166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.755251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:35.760921 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 972/1952] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:35 2022] PUT /identity/v3/projects/2cfd66dfb54248258c66e23eda6084c9/users/bc8d5aec013b4ca894ff14c834d9a6a2/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:35.773052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.773349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.773634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.805246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] MFA Rules not processed for user `bc8d5aec013b4ca894ff14c834d9a6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:35.806545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:35.856964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:35.880910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 981/1953] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:35.892757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:35.894524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:35.958798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2557-ba5d-4342-94cb-4667b7a83b51 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:35.963800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20765f-8bf8-4ca3-815b-960531895aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:35.964596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20765f-8bf8-4ca3-815b-960531895aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:35.965097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20765f-8bf8-4ca3-815b-960531895aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:35.965709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20765f-8bf8-4ca3-815b-960531895aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:35.976528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20765f-8bf8-4ca3-815b-960531895aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:35.978557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20765f-8bf8-4ca3-815b-960531895aa2 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:35.990231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 973/1954] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:36.014512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:36.109673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612ce26-b827-4c78-99c5-d816eee91846 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:36.114762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:36.115679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:36.115998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:36.116281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] PATH_INFO: `/v3/users/bc8d5aec013b4ca894ff14c834d9a6a2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:36.118147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8d5aec013b4ca894ff14c834d9a6a2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:36.121059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:36.179449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8d5aec013b4ca894ff14c834d9a6a2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.180436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8d5aec013b4ca894ff14c834d9a6a2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.189550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8d5aec013b4ca894ff14c834d9a6a2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.190292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8d5aec013b4ca894ff14c834d9a6a2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.198297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8d5aec013b4ca894ff14c834d9a6a2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.216097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 982/1955] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:36 2022] DELETE /identity/v3/users/bc8d5aec013b4ca894ff14c834d9a6a2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:36.220508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:36.220918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:36.221344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:36.238830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:36.239997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:36.300942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 974/1956] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:36.311141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:36.312572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:36.378149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469223c-c0e1-47a0-ab4e-90c61b66404e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:36.381955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e71f76-53c1-4003-8e5d-6c4f6dd8d322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:36.382644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e71f76-53c1-4003-8e5d-6c4f6dd8d322 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:36.382960 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e71f76-53c1-4003-8e5d-6c4f6dd8d322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:36.383242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e71f76-53c1-4003-8e5d-6c4f6dd8d322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:36.394374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e71f76-53c1-4003-8e5d-6c4f6dd8d322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:36.395856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e71f76-53c1-4003-8e5d-6c4f6dd8d322 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:36.405588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 983/1957] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:36.827779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:36.829326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:36.846527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18eb38-b81d-4ec8-ad5d-add3bc2e3863 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:36.859545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:36.861136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:36.861625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:36.862071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] PATH_INFO: `/v3/projects/2cfd66dfb54248258c66e23eda6084c9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:36.869056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfd66dfb54248258c66e23eda6084c9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:36.871389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:36.942804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfd66dfb54248258c66e23eda6084c9', 'request_id': 'req-0837c6c9-864a-4f12-b4a8-d4984329c0b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.950598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfd66dfb54248258c66e23eda6084c9', 'request_id': 'req-0837c6c9-864a-4f12-b4a8-d4984329c0b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.958756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfd66dfb54248258c66e23eda6084c9', 'request_id': 'req-0837c6c9-864a-4f12-b4a8-d4984329c0b8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.976346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] The token cache is being invalidate because project 2cfd66dfb54248258c66e23eda6084c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:36.976346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.977066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0837c6c9-864a-4f12-b4a8-d4984329c0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:36.982898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 975/1958] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:36 2022] DELETE /identity/v3/projects/2cfd66dfb54248258c66e23eda6084c9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:36.985632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:36.986253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:36.987015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.012753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:37.029547 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:37.031209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:37.094347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 984/1959] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:37.102189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.102532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.102818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.121256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:37.122477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:37.130794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 976/1960] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:37.135934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.145572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2fc97-7c4a-4cd1-86e3-880e12e17c80 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.150801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.151480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.151767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.152049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.152770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:37.153079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.154677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.166182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 985/1961] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:37.170446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.179714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca26d62-d3c2-4194-a5e3-940c338b1f34 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.184531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.185148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.185477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.185758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.186447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.188432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.209263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:37.219398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 977/1962] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:37 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:37.223431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.234394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2d0ca-156f-4d6d-862d-2e3fe47bec9b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.303927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.304977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.305261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.306036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.306743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.309700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.336223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 986/1963] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:37 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:37.344076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.354950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54821-cb92-4260-b08c-174274f542e9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.360202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.361166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.361550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.361897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.362644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.364321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.376485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 978/1964] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:37.380620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.394472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d761416-442d-41a3-a792-35fa19e4a110 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.400532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a578f-7b2e-4da8-bfb3-182b978a6483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.401421 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a578f-7b2e-4da8-bfb3-182b978a6483 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.401830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a578f-7b2e-4da8-bfb3-182b978a6483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.402325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a578f-7b2e-4da8-bfb3-182b978a6483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.403328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a578f-7b2e-4da8-bfb3-182b978a6483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.405086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a578f-7b2e-4da8-bfb3-182b978a6483 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.416617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 987/1965] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:37.426447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.437722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbcef1-c5da-4236-8b42-f45bc0ea1a3b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.443397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.444289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.444580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.444864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] PATH_INFO: `/v3/projects/8b30614902a44689883603399bb1478b/users/a2ba5a807e014cceb812ffc2662de110/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.467709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b30614902a44689883603399bb1478b, user_id=a2ba5a807e014cceb812ffc2662de110, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.470613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.489796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 979/1966] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:37 2022] PUT /identity/v3/projects/8b30614902a44689883603399bb1478b/users/a2ba5a807e014cceb812ffc2662de110/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:37.502002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.502303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.502591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.560718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] MFA Rules not processed for user `a2ba5a807e014cceb812ffc2662de110`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:37.562036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:37.666290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 988/1967] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:37.682670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:37.684561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:37.766907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4f200-96db-4ca4-941b-7f129330d757 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:37.770661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c6df-7ff3-4eef-abac-dcd9ab126c44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:37.771415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c6df-7ff3-4eef-abac-dcd9ab126c44 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:37.771764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c6df-7ff3-4eef-abac-dcd9ab126c44 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:37.772116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c6df-7ff3-4eef-abac-dcd9ab126c44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:37.782093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c6df-7ff3-4eef-abac-dcd9ab126c44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:37.783740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c6df-7ff3-4eef-abac-dcd9ab126c44 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:37.793014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 980/1968] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:38.763380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:38.764801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:38.769910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:38.862434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7828ea4-9d6a-491b-a9e0-17c2c88a9981 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:38.867235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:38.868655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:38.868942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:38.869343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] PATH_INFO: `/v3/users/b33b5828648040a5a92477af9ae49138` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:38.870575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33b5828648040a5a92477af9ae49138)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:38.872597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:38.944460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33b5828648040a5a92477af9ae49138'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:38.945221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33b5828648040a5a92477af9ae49138'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:38.958370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33b5828648040a5a92477af9ae49138'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:38.959290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33b5828648040a5a92477af9ae49138'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:38.971394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33b5828648040a5a92477af9ae49138'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:39.010948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 989/1969] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:38 2022] DELETE /identity/v3/users/b33b5828648040a5a92477af9ae49138 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:39.017412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:39.017957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:39.017957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:39.068433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:39.074887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:39.221283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 981/1970] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:39.247162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:39.248946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:39.343374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d0d73-76dd-458c-94fd-f9171d549eca admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:39.348122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:39.349310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:39.349801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:39.350300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:39.366017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:39.370785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:39.383445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 990/1971] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:40.063091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:40.064656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:40.077757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8adba2-34fa-431c-aa3e-3892312352fd None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:40.082793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:40.083885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:40.084430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:40.084853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] PATH_INFO: `/v3/projects/0eb6ad7579554793b653c93a71fa9ff4` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:40.087476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb6ad7579554793b653c93a71fa9ff4)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:40.089874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:40.206169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb6ad7579554793b653c93a71fa9ff4', 'request_id': 'req-6db9004b-0518-4def-b687-f3fdef8a9592'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.218183 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb6ad7579554793b653c93a71fa9ff4', 'request_id': 'req-6db9004b-0518-4def-b687-f3fdef8a9592'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.221354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:40.226822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb6ad7579554793b653c93a71fa9ff4', 'request_id': 'req-6db9004b-0518-4def-b687-f3fdef8a9592'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.248727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] The token cache is being invalidate because project 0eb6ad7579554793b653c93a71fa9ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:40.249186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.249869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.254683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 982/1972] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:40 2022] DELETE /identity/v3/projects/0eb6ad7579554793b653c93a71fa9ff4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:40.255758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:40.265743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7cf2e-9b60-402e-a442-e03694849228 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:40.288083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:40.309347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:40.310918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:40.353326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:40.354184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:40.354469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:40.354750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] PATH_INFO: `/v3/users/4af9334154c14442b47d3e9f64bf02b2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:40.356088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af9334154c14442b47d3e9f64bf02b2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:40.358336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:40.395956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9004b-0518-4def-b687-f3fdef8a9592 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:40.402257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:40.403277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:40.403640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:40.404018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] PATH_INFO: `/v3/users/eda2e30bdf5f430aaa71818dcf02cb34` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:40.405391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda2e30bdf5f430aaa71818dcf02cb34)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:40.407486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:40.419260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af9334154c14442b47d3e9f64bf02b2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.419952 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af9334154c14442b47d3e9f64bf02b2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.429239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af9334154c14442b47d3e9f64bf02b2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.429943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af9334154c14442b47d3e9f64bf02b2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.440041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af9334154c14442b47d3e9f64bf02b2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.466123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda2e30bdf5f430aaa71818dcf02cb34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.466329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 991/1973] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:40 2022] DELETE /identity/v3/users/4af9334154c14442b47d3e9f64bf02b2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:40.467248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda2e30bdf5f430aaa71818dcf02cb34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.468007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda2e30bdf5f430aaa71818dcf02cb34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.475715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda2e30bdf5f430aaa71818dcf02cb34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.492787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda2e30bdf5f430aaa71818dcf02cb34'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:40.509172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 983/1974] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:40 2022] DELETE /identity/v3/users/eda2e30bdf5f430aaa71818dcf02cb34 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:40.912520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:40.915988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.068102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.069928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.108472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c62350-77af-447a-96db-17d2b885484a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.191682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ebac4-f614-4d11-8a95-9d63d17ece3f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.197526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.198635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.198994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.199282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] PATH_INFO: `/v3/projects/1dc929b713a94027b5584e89484ef548` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.200511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc929b713a94027b5584e89484ef548)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.202210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:41.202457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.220068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.221851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.222329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.223341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] PATH_INFO: `/v3/projects/d4ed0448570344758cc1f516eef58ac9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.224657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:41.227480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ed0448570344758cc1f516eef58ac9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.229462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.290455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc929b713a94027b5584e89484ef548', 'request_id': 'req-3a514139-443f-45f3-8326-00acd64ad442'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.306771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc929b713a94027b5584e89484ef548', 'request_id': 'req-3a514139-443f-45f3-8326-00acd64ad442'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.321001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ed0448570344758cc1f516eef58ac9', 'request_id': 'req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.323243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc929b713a94027b5584e89484ef548', 'request_id': 'req-3a514139-443f-45f3-8326-00acd64ad442'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.332829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ed0448570344758cc1f516eef58ac9', 'request_id': 'req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.343148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ed0448570344758cc1f516eef58ac9', 'request_id': 'req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.347566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] The token cache is being invalidate because project 1dc929b713a94027b5584e89484ef548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:41.348056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.349195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a514139-443f-45f3-8326-00acd64ad442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.351269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 984/1975] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:41 2022] DELETE /identity/v3/projects/1dc929b713a94027b5584e89484ef548 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:41.357894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778394c-b5e9-4393-98e4-1057367ca6ed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.358263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778394c-b5e9-4393-98e4-1057367ca6ed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.358767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778394c-b5e9-4393-98e4-1057367ca6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.361591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] The token cache is being invalidate because project d4ed0448570344758cc1f516eef58ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:41.361898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.362553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.367095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 992/1976] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:40 2022] DELETE /identity/v3/projects/d4ed0448570344758cc1f516eef58ac9 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:41.369129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.373033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.396608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2778394c-b5e9-4393-98e4-1057367ca6ed None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:41.398240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2778394c-b5e9-4393-98e4-1057367ca6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.467234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e26b1-c7c7-4aac-866f-c9a6b67b82dd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.473404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.474861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.475259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.475663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] PATH_INFO: `/v3/users/136054ae6b7f4e788c5e11d13e246ef3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.476995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] RBAC: Authorizing `identity:delete_user(user_id=136054ae6b7f4e788c5e11d13e246ef3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.479672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.491622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 985/1977] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:41.500016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.500640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.501436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.523928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:41.525419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.535779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 986/1978] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:41.539708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.544252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136054ae6b7f4e788c5e11d13e246ef3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.544913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136054ae6b7f4e788c5e11d13e246ef3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.551059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cdff2-1bd2-494f-9c78-cd7a0f88e5f5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.553125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136054ae6b7f4e788c5e11d13e246ef3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.553788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136054ae6b7f4e788c5e11d13e246ef3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.556801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.557761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.558140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.558509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.559370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:41.559855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.561677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.567009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136054ae6b7f4e788c5e11d13e246ef3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:41.579373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 987/1979] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:04:41.580593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.582465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.583957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:41.590927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 993/1980] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:41 2022] DELETE /identity/v3/users/136054ae6b7f4e788c5e11d13e246ef3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:41.600635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b63352-95da-4ec2-bd86-540add2ee09a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.699173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.700233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.700492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.700923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.701769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.704333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.753208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 988/1981] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:41 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:41.757884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.771273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfdd0b-768f-459f-9b43-daf90ce812af admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.809932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.812585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:41.822528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:41.864610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.865296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.866875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.866875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.866875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.869869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.901992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 994/1982] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:41 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:41.902987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.914052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca14f70-6bfc-473e-953e-b193b9bc2c4f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.914570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760350c4-d3f1-455b-babc-a0ce049d90e3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.922684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.922833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.923414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.923697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.923975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.924068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.924424 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.924704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.924879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] PATH_INFO: `/v3/projects/e5cff105a1254084b082457fa7ca3cf6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.926334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.926716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5cff105a1254084b082457fa7ca3cf6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.928910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.938051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 995/1983] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:41.941903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.953436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22d3fd-32d6-4df3-aab2-dfdac65bd19f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.958619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:41.959422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:41.959717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:41.960000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:41.961479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:41.963795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:41.975895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 996/1984] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:04:41.980148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:41.991041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785c40-9f9c-451f-853d-9643f0791322 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:41.997561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5cff105a1254084b082457fa7ca3cf6', 'request_id': 'req-c351d57c-cc7b-455b-a97f-6b184a3368f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:42.023089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5cff105a1254084b082457fa7ca3cf6', 'request_id': 'req-c351d57c-cc7b-455b-a97f-6b184a3368f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:42.031597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5cff105a1254084b082457fa7ca3cf6', 'request_id': 'req-c351d57c-cc7b-455b-a97f-6b184a3368f0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:42.049715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] The token cache is being invalidate because project e5cff105a1254084b082457fa7ca3cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:42.050101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:42.051008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c351d57c-cc7b-455b-a97f-6b184a3368f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:42.052847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 989/1985] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:41 2022] DELETE /identity/v3/projects/e5cff105a1254084b082457fa7ca3cf6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:04:42.058409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d9a7b-1925-444b-854f-e0f4f2865f74 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.058803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d9a7b-1925-444b-854f-e0f4f2865f74 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.059598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d9a7b-1925-444b-854f-e0f4f2865f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.062532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd6b91-cf33-4418-abef-e31c9cc9d712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.063470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6b91-cf33-4418-abef-e31c9cc9d712 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.063761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6b91-cf33-4418-abef-e31c9cc9d712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.064047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6b91-cf33-4418-abef-e31c9cc9d712 admin admin] PATH_INFO: `/v3/projects/a1b303a5b7f24930acdd0b05d6ed8e91/users/aaf45acacace44eba6d9e73cfdfe4a28/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.081792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6b91-cf33-4418-abef-e31c9cc9d712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b303a5b7f24930acdd0b05d6ed8e91, user_id=aaf45acacace44eba6d9e73cfdfe4a28, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.083432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-999d9a7b-1925-444b-854f-e0f4f2865f74 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:42.084610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6b91-cf33-4418-abef-e31c9cc9d712 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.084871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-999d9a7b-1925-444b-854f-e0f4f2865f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:42.096874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 990/1986] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:42.100770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 997/1987] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:41 2022] PUT /identity/v3/projects/a1b303a5b7f24930acdd0b05d6ed8e91/users/aaf45acacace44eba6d9e73cfdfe4a28/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:42.116796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.117054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.117156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.117256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.117439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.117541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.144199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:42.145755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:42.158090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] MFA Rules not processed for user `aaf45acacace44eba6d9e73cfdfe4a28`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:42.159231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:42.265367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:42.300955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 998/1988] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:42.310091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 991/1989] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:42.314537 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.317095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.318878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:42.335833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ba73-bf2d-47fa-84f3-5f46ca5ea4d0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.349400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.351664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.353119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.354890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.357269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:42.359121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.363323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.406226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 999/1990] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:42.409729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.422308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:42.450172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb263ca5-b8d3-4834-90b4-0e4014df46a9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.466551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.467349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.467635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.467915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.468766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.471230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.482675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62829487-4e2a-4315-a6cf-ee78f069b5ec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.487376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.488451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.488885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.489264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.503836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.505480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.531172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 992/1991] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:42.534248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1000/1992] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:42 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:42.539747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.565244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0f183-5b69-49df-8674-0dd6717c5c74 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.740511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.741231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.741516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.741845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.742550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.745319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.787992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 993/1993] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:42 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:42.792321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.808977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91bb5-e562-4d7b-80f6-c358d8990db0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.824599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.825453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.825806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.826132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.827559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.829568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.865329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1001/1994] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:42.870704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.885262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cf54b-26af-464c-863a-450cca2d324f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.890676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f888604-ec5e-4b68-a2bb-0222243f274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.891389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f888604-ec5e-4b68-a2bb-0222243f274b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.891681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f888604-ec5e-4b68-a2bb-0222243f274b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.892022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f888604-ec5e-4b68-a2bb-0222243f274b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.892788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f888604-ec5e-4b68-a2bb-0222243f274b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.894413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f888604-ec5e-4b68-a2bb-0222243f274b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:42.913699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 994/1995] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:42.918500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:42.935598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4a2d-beb0-47f9-8cd4-95a458527f33 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:42.945147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:42.946943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:42.947650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:42.948373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] PATH_INFO: `/v3/projects/194092b39e5349ccadb69083f0007632/users/2e4abe9465c24510ab16263806439c40/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:42.967490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=194092b39e5349ccadb69083f0007632, user_id=2e4abe9465c24510ab16263806439c40, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:42.984745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:43.040661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:43.041107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:43.041538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:43.043821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1002/1996] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:42 2022] PUT /identity/v3/projects/194092b39e5349ccadb69083f0007632/users/2e4abe9465c24510ab16263806439c40/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:43.082569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] MFA Rules not processed for user `2e4abe9465c24510ab16263806439c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:43.083855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:43.164766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 995/1997] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:43.176993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:43.237175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8440e-edf3-474f-8aaa-a46324e37d1f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:43.241846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b137d3-0f48-498e-bdc4-f0486f1ca091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:43.242501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b137d3-0f48-498e-bdc4-f0486f1ca091 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:43.242788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b137d3-0f48-498e-bdc4-f0486f1ca091 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:43.243512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b137d3-0f48-498e-bdc4-f0486f1ca091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:43.252717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b137d3-0f48-498e-bdc4-f0486f1ca091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:43.254197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b137d3-0f48-498e-bdc4-f0486f1ca091 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:43.263675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1003/1998] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:43.284441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:43.329677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:43.351784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1fada-f5c6-42e5-8eea-c15f69b6808a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:43.356680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:43.357634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:43.358068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:43.358394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] PATH_INFO: `/v3/users/2e4abe9465c24510ab16263806439c40` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:43.359696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4abe9465c24510ab16263806439c40)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:43.361734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:43.412177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4abe9465c24510ab16263806439c40'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:43.412897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4abe9465c24510ab16263806439c40'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:43.413638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4abe9465c24510ab16263806439c40'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:43.422002 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4abe9465c24510ab16263806439c40'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:43.436602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4abe9465c24510ab16263806439c40'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:43.453871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 996/1999] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:04:43 2022] DELETE /identity/v3/users/2e4abe9465c24510ab16263806439c40 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:43.456087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:43.456787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:43.456787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:43.478196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:43.479392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:43.557984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1004/2000] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:43.567826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:43.569581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:43.634972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fe16c-92c2-41fa-8e46-e67c0610b2e1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:43.639477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b27b8-87bf-4b15-adf2-fd6dc97407c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:43.640146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b27b8-87bf-4b15-adf2-fd6dc97407c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:43.640431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b27b8-87bf-4b15-adf2-fd6dc97407c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:43.640707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b27b8-87bf-4b15-adf2-fd6dc97407c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:43.651878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b27b8-87bf-4b15-adf2-fd6dc97407c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:43.653336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b27b8-87bf-4b15-adf2-fd6dc97407c8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:43.675158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 997/2001] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:44.188578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.190181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:44.204636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd22b4c-7c84-407d-92ab-156da3684aae None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.210995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.212522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.213449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.215245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] PATH_INFO: `/v3/projects/194092b39e5349ccadb69083f0007632` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.219897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=194092b39e5349ccadb69083f0007632)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.222508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.232399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:44.285678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194092b39e5349ccadb69083f0007632', 'request_id': 'req-d104da10-eb82-43a6-a231-a1fbd098ab0c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:44.292441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194092b39e5349ccadb69083f0007632', 'request_id': 'req-d104da10-eb82-43a6-a231-a1fbd098ab0c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:44.299560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194092b39e5349ccadb69083f0007632', 'request_id': 'req-d104da10-eb82-43a6-a231-a1fbd098ab0c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:44.312377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] The token cache is being invalidate because project 194092b39e5349ccadb69083f0007632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:04:44.312744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:44.313469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d104da10-eb82-43a6-a231-a1fbd098ab0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:04:44.317534 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1005/2002] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:04:44 2022] DELETE /identity/v3/projects/194092b39e5349ccadb69083f0007632 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:44.321884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.322539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.322539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.343799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:44.345026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:44.432020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 998/2003] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:44.440602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.440899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.441192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.463895 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:44.465174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:44.473829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1006/2004] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:44.480199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.486302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:44.493301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca2f1d-e3c7-462c-97f0-4e8282d72f89 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.500816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.501515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.501801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.502080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.502806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:04:44.503149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.504791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.520968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 999/2005] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:04:44.525568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.535565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3ac80-a1f1-4d46-b3f5-d89a82542f3d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.540855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.541797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.542044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.542326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.543066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.545457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.587322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1007/2006] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:04:44 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:04:44.592170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.604228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd621-812a-47df-9743-8209efcf4ff9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.681026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.681810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.682027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.682256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.682985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.684938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.710382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1000/2007] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:04:44 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:04:44.714050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.724824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca93bf8-eb79-455f-8f13-a47e62dddc6e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.730159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.730921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.733256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.733540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.734278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.735929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.748856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1008/2008] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:44.752752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.762826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c685e5f-02ee-43a8-b2d9-0c0fafaf7509 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.768259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0df94a-acb0-4eaf-8487-dc429cb3f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.768935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df94a-acb0-4eaf-8487-dc429cb3f690 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.769227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df94a-acb0-4eaf-8487-dc429cb3f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.769505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df94a-acb0-4eaf-8487-dc429cb3f690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.770225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0df94a-acb0-4eaf-8487-dc429cb3f690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.771845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0df94a-acb0-4eaf-8487-dc429cb3f690 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.782812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1001/2009] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:04:44.787019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.797728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95670e9b-0bf9-4765-a60d-818820c83ee5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:44.802301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:44.803220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.803618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.803901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] PATH_INFO: `/v3/projects/c37ecaa897a7485fb4a2ccdce8496eb9/users/05d51a9c4d8e4d869252835c37764abe/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.819154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37ecaa897a7485fb4a2ccdce8496eb9, user_id=05d51a9c4d8e4d869252835c37764abe, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:44.822155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:44.834664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1009/2010] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:04:44 2022] PUT /identity/v3/projects/c37ecaa897a7485fb4a2ccdce8496eb9/users/05d51a9c4d8e4d869252835c37764abe/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:04:44.840252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854b5b-077d-4248-879f-a072e9b4355a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:44.840547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854b5b-077d-4248-879f-a072e9b4355a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:44.841133 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854b5b-077d-4248-879f-a072e9b4355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:44.860112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-08854b5b-077d-4248-879f-a072e9b4355a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:44.861307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08854b5b-077d-4248-879f-a072e9b4355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:44.921652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1002/2011] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:44.935469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:44.939988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:45.055477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ef82d-20d3-4519-9b35-2c2b3a584173 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:45.066529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:45.067364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:45.067726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:45.068061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:45.082698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:45.084785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:45.097589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1010/2012] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:45.945061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f15c9-1f6d-4b6d-8556-8a12441d494b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:45.945672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f15c9-1f6d-4b6d-8556-8a12441d494b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:45.945672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f15c9-1f6d-4b6d-8556-8a12441d494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:45.975618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ae4f15c9-1f6d-4b6d-8556-8a12441d494b None None] MFA Rules not processed for user `05d51a9c4d8e4d869252835c37764abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:45.976878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ae4f15c9-1f6d-4b6d-8556-8a12441d494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:46.051861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:46.066317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1003/2013] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:04:46.077400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:04:46.079757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:46.136063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:46.156328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554441-cfb6-4f9e-9db0-902248014df1 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:04:46.159941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:04:46.160598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:46.160882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:46.161158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:46.171671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:04:46.173165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:04:46.182241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1011/2014] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:04:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:04:47.721020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:04:47.721820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:04:47.722486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:04:47.759962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] MFA Rules not processed for user `05d51a9c4d8e4d869252835c37764abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:04:47.761269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:04:47.776473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1004/2015] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:04:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:04:56.178757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=10 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:04:57.751113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:07.236621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:07.238347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:07.254047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcd13c-2a30-459f-bd42-bfc811fd1258 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:07.258634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:07.259644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.260002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.260274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] PATH_INFO: `/v3/users/3ba79450153f4399bfe9d3900ed12523` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.261497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba79450153f4399bfe9d3900ed12523)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:07.263481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:07.315275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba79450153f4399bfe9d3900ed12523'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.316000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba79450153f4399bfe9d3900ed12523'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.323687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba79450153f4399bfe9d3900ed12523'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.324584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba79450153f4399bfe9d3900ed12523'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.333656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13deb5bc-c6ab-4075-a4c0-6c5f5a5607dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba79450153f4399bfe9d3900ed12523'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.354400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1012/2016] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:07 2022] DELETE /identity/v3/users/3ba79450153f4399bfe9d3900ed12523 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:07.531854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:07.548524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d594e-211c-402e-9eb3-acf0f2cbc7aa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:07.609435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:07.610250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.610559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.610987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] PATH_INFO: `/v3/projects/30f4ac789e8e41468a98175ef8d60d18` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.612314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f4ac789e8e41468a98175ef8d60d18)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:07.614582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:07.663783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f4ac789e8e41468a98175ef8d60d18', 'request_id': 'req-5de9ad26-5eb9-4852-99d6-8c73dba94f88'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.669375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f4ac789e8e41468a98175ef8d60d18', 'request_id': 'req-5de9ad26-5eb9-4852-99d6-8c73dba94f88'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.675321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f4ac789e8e41468a98175ef8d60d18', 'request_id': 'req-5de9ad26-5eb9-4852-99d6-8c73dba94f88'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.687675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] The token cache is being invalidate because project 30f4ac789e8e41468a98175ef8d60d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:07.687974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.688635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5de9ad26-5eb9-4852-99d6-8c73dba94f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:07.690250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1005/2017] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:07 2022] DELETE /identity/v3/projects/30f4ac789e8e41468a98175ef8d60d18 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:07.695541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.695857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.696153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.714583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:07.715834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:07.778894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1013/2018] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:07.787530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.787826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.788130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.808012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:07.809383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:07.819785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1006/2019] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:07.824918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:07.835754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cabad-7f6f-45b8-a572-73154aef355f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:07.841588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:07.842527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.842813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.843266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.844038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:05:07.844384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:07.846277 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:07.860449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1014/2020] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:05:07.865375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:07.875999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a9db0-4481-42c2-8b47-d57061c7d22e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:07.880680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:07.881467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.881753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.882031 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.882717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:07.885329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:07.911512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1007/2021] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:07 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:07.916790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:07.929589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731c52-9d31-418b-8b41-f1508ff31cca admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:07.989033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:07.989715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:07.990005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:07.990289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:07.991012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:07.992961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:08.000143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:08.015999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1015/2022] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:07 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:08.020240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:08.030862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d228db-b0f5-4343-8302-68b5e219be16 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:08.035512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:08.036163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:08.036451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:08.036733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:08.037445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:08.039129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:08.051121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1008/2023] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:05:08.055256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:08.066388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8518711-be1b-409e-bdf2-56db464b7393 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:08.071453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce972b-2319-4d44-ba7a-7b827c691d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:08.072757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce972b-2319-4d44-ba7a-7b827c691d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:08.073075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce972b-2319-4d44-ba7a-7b827c691d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:08.073398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce972b-2319-4d44-ba7a-7b827c691d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:08.074121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce972b-2319-4d44-ba7a-7b827c691d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:08.075764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce972b-2319-4d44-ba7a-7b827c691d41 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:08.085825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1016/2024] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:08.089958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:08.100346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f730c-c0a7-4551-9051-2ba9ba011815 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:08.105232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:08.106596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:08.106918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:08.107198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] PATH_INFO: `/v3/projects/29f804f9e1c94dbc957f1db38e034ec3/users/d2e3406e71464f0bb0b6202ce1719eb5/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:08.123233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f804f9e1c94dbc957f1db38e034ec3, user_id=d2e3406e71464f0bb0b6202ce1719eb5, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:08.125879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:08.155039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1009/2025] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:08 2022] PUT /identity/v3/projects/29f804f9e1c94dbc957f1db38e034ec3/users/d2e3406e71464f0bb0b6202ce1719eb5/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:08.166134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:08.166503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:08.166983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:08.206045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] MFA Rules not processed for user `d2e3406e71464f0bb0b6202ce1719eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:08.207366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:08.302026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1017/2026] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:08.313228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:08.315066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:08.357480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:08.396156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543c9f-5a8c-4a0b-abc5-5cf3d59f975d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:08.400684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2e967-21ca-44a1-a17e-47c7e0a2e92c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:08.402078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2e967-21ca-44a1-a17e-47c7e0a2e92c service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:08.402580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2e967-21ca-44a1-a17e-47c7e0a2e92c service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:08.402969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2e967-21ca-44a1-a17e-47c7e0a2e92c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:08.412409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2e967-21ca-44a1-a17e-47c7e0a2e92c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:08.413967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2e967-21ca-44a1-a17e-47c7e0a2e92c service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:08.426028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1010/2027] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:14.895236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:14.896670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:14.981372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a9feb-78b2-4828-bcc1-fb685b689177 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:14.986188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:14.987236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:14.987548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:14.987896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] PATH_INFO: `/v3/users/a2ba5a807e014cceb812ffc2662de110` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:14.989242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ba5a807e014cceb812ffc2662de110)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:14.991488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:15.046099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ba5a807e014cceb812ffc2662de110'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.046580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ba5a807e014cceb812ffc2662de110'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.054525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ba5a807e014cceb812ffc2662de110'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.055268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ba5a807e014cceb812ffc2662de110'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.064030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ba5a807e014cceb812ffc2662de110'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.070227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:15.088646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1018/2028] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:14 2022] DELETE /identity/v3/users/a2ba5a807e014cceb812ffc2662de110 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:15.093463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:15.094014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:15.094014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:15.112869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:15.114256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:15.186649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1011/2029] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:15.196309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:15.197859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:15.283571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516ca57-5f6f-4946-9a37-681dca4d8b23 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:15.287611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:15.288461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:15.288750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:15.289030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:15.299290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:15.300726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:15.311190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1019/2030] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:15.713688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:15.716502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:15.749339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce33f96-d85b-47c0-9dc1-8261e9050f7a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:15.769775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:15.770981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:15.771371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:15.771742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] PATH_INFO: `/v3/projects/8b30614902a44689883603399bb1478b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:15.773326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b30614902a44689883603399bb1478b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:15.776190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:15.839059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b30614902a44689883603399bb1478b', 'request_id': 'req-cca30e1a-4172-4d5e-b4e8-404559f45bb8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.847633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b30614902a44689883603399bb1478b', 'request_id': 'req-cca30e1a-4172-4d5e-b4e8-404559f45bb8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.856770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b30614902a44689883603399bb1478b', 'request_id': 'req-cca30e1a-4172-4d5e-b4e8-404559f45bb8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.884692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] The token cache is being invalidate because project 8b30614902a44689883603399bb1478b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:15.885086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.885960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cca30e1a-4172-4d5e-b4e8-404559f45bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:15.894387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1012/2031] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:15 2022] DELETE /identity/v3/projects/8b30614902a44689883603399bb1478b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:19.375838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:19.377595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:19.391352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02484141-7f85-46c8-a0f4-34f722c107cc service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:19.449669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:19.450602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:19.450828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:19.451156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] PATH_INFO: `/v3/users/d2e3406e71464f0bb0b6202ce1719eb5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:19.452356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e3406e71464f0bb0b6202ce1719eb5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:19.454320 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:19.503030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e3406e71464f0bb0b6202ce1719eb5'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:19.503611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e3406e71464f0bb0b6202ce1719eb5'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:19.511099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e3406e71464f0bb0b6202ce1719eb5'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:19.511730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e3406e71464f0bb0b6202ce1719eb5'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:19.518963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e3406e71464f0bb0b6202ce1719eb5'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:19.536062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1020/2032] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:19 2022] DELETE /identity/v3/users/d2e3406e71464f0bb0b6202ce1719eb5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:19.540760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:19.541385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:19.541385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:19.559632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:19.560842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:19.613754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:19.628045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1013/2033] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:19.637627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:19.639118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:19.650094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:19.699702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73997d-e434-49b7-9fd3-2eeb2489c04f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:19.703391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5078fe8-b9ad-49a5-b78c-b936454f0e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:19.704042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5078fe8-b9ad-49a5-b78c-b936454f0e4a service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:19.704356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5078fe8-b9ad-49a5-b78c-b936454f0e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:19.704666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5078fe8-b9ad-49a5-b78c-b936454f0e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:19.714007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5078fe8-b9ad-49a5-b78c-b936454f0e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:19.715493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5078fe8-b9ad-49a5-b78c-b936454f0e4a service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:19.725165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1021/2034] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:20.006303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.006303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:20.029189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ae141-56b8-45ee-a3e3-ecf1f392185b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.049475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.050549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.050873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.051196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] PATH_INFO: `/v3/projects/29f804f9e1c94dbc957f1db38e034ec3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.052712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=29f804f9e1c94dbc957f1db38e034ec3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.055382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.059103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b143772-673e-42d4-80aa-ad24930cec35 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.059722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b143772-673e-42d4-80aa-ad24930cec35 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.060056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b143772-673e-42d4-80aa-ad24930cec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.078434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7b143772-673e-42d4-80aa-ad24930cec35 None None] MFA Rules not processed for user `05d51a9c4d8e4d869252835c37764abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:20.079616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7b143772-673e-42d4-80aa-ad24930cec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:20.108726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29f804f9e1c94dbc957f1db38e034ec3', 'request_id': 'req-c348e0e8-75e1-456a-8c97-b840bf456aea'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:20.114093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29f804f9e1c94dbc957f1db38e034ec3', 'request_id': 'req-c348e0e8-75e1-456a-8c97-b840bf456aea'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:20.120630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29f804f9e1c94dbc957f1db38e034ec3', 'request_id': 'req-c348e0e8-75e1-456a-8c97-b840bf456aea'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:20.133430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] The token cache is being invalidate because project 29f804f9e1c94dbc957f1db38e034ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:20.133947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:20.134681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c348e0e8-75e1-456a-8c97-b840bf456aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:20.136287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1014/2035] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:20 2022] DELETE /identity/v3/projects/29f804f9e1c94dbc957f1db38e034ec3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:20.140967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1022/2036] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:20.141404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.141932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.142404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.162655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:20.163925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:20.227340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1015/2037] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:20.235458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.235855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.236217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.260509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:20.261747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:20.270089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1023/2038] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:20.275613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.285946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f3b31-80a6-4140-8d0c-fc0bfca59072 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.290651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.291359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.291653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.291939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.292670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:05:20.292983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.294600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.309095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1016/2039] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:05:20.313583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.324521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ff7d-03f1-49fd-bde3-5b27a0e95fdf None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.330338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.330977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.331273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.331548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.332235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.334163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.368006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1024/2040] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:20.369392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.380633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae1fba-6f9f-49df-bb3b-e2d5132c3f42 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.441802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.442493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.442780 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.443164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.443868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.445842 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.456643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:20.471599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1017/2041] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:20.475316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.487998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7420d-8743-4d5a-9df2-445da76d0849 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.494659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.495377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.495670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.495963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.496829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.498673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.519272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1025/2042] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:05:20.524192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.538104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdefe0-d2b2-4460-b316-feeefadea93f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.546948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdedd3-d20d-4ed0-ac64-63c67e24da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.548064 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdedd3-d20d-4ed0-ac64-63c67e24da49 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.548460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdedd3-d20d-4ed0-ac64-63c67e24da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.548848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdedd3-d20d-4ed0-ac64-63c67e24da49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.549726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdedd3-d20d-4ed0-ac64-63c67e24da49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.551526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdedd3-d20d-4ed0-ac64-63c67e24da49 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.562916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1018/2043] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:20.568657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.579887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b97f7-2081-4d2c-935f-8bcdf15d4483 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.584788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.585619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.585903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.586183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] PATH_INFO: `/v3/projects/89f823d212244767a0b83e6779269cf6/users/0ac1e691aa63436e9e74de615f653749/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.602073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89f823d212244767a0b83e6779269cf6, user_id=0ac1e691aa63436e9e74de615f653749, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.605460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.620653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1026/2044] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:20 2022] PUT /identity/v3/projects/89f823d212244767a0b83e6779269cf6/users/0ac1e691aa63436e9e74de615f653749/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:20.632271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.632569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.632853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.662455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] MFA Rules not processed for user `0ac1e691aa63436e9e74de615f653749`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:20.663784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:20.747511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1019/2045] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:20.751995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.807682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:20.819256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e5c2-5d20-4d0b-87d1-5eb9b1f9fdf9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.825716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.826559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.827044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.827514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.828909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.831078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.864805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1027/2046] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:20.865960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.877338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f9bf0-909b-4143-87ca-1c23db7148c5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:20.951838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:20.952594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:20.953013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:20.953302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:20.954143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:20.956201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:20.982905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1020/2047] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:20 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:05:20.986149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:20.997505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e163f-eac8-49d6-88f5-1048d9de1f0e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.006170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.006170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.006170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.006170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.006582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:21.014471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:21.030241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1028/2048] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:21.034188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:21.044858 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc08e-0f76-4bb6-a13c-72ccabfb91df admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.050463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.051570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.051851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.052137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] PATH_INFO: `/v3/projects/77ad7d78cb9848339d1c47142b0eb696/users/b07741297e4e462396a217abc2a3e4d9/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.073648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ad7d78cb9848339d1c47142b0eb696, user_id=b07741297e4e462396a217abc2a3e4d9, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:21.076592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:21.109674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1021/2049] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:21 2022] PUT /identity/v3/projects/77ad7d78cb9848339d1c47142b0eb696/users/b07741297e4e462396a217abc2a3e4d9/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:05:21.110862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:21.121997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ebb94-5495-48c1-bb3d-2c6428b3e3bc admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.201922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a156a5-918b-4baa-85ea-953cc9c2ff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.202599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a156a5-918b-4baa-85ea-953cc9c2ff99 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.203048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a156a5-918b-4baa-85ea-953cc9c2ff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.203333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a156a5-918b-4baa-85ea-953cc9c2ff99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.204070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a156a5-918b-4baa-85ea-953cc9c2ff99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:21.205700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a156a5-918b-4baa-85ea-953cc9c2ff99 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:21.219121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1029/2050] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:05:21.223018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:21.233470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf12a45-c09a-4321-8b77-c26237722bd4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.238204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.239143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.239457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.239743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] PATH_INFO: `/v3/projects/77ad7d78cb9848339d1c47142b0eb696/users/b07741297e4e462396a217abc2a3e4d9/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.241735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ad7d78cb9848339d1c47142b0eb696, user_id=b07741297e4e462396a217abc2a3e4d9, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:21.244506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:21.255673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1022/2051] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:21 2022] PUT /identity/v3/projects/77ad7d78cb9848339d1c47142b0eb696/users/b07741297e4e462396a217abc2a3e4d9/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:21.267273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.267678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.267997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.296896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] MFA Rules not processed for user `b07741297e4e462396a217abc2a3e4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:21.298156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:21.377193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1030/2052] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:21.387823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:21.389516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:21.422062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:21.454454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a12f35-d1b2-4803-919c-8f6ed02c116b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.458432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd94891-eb35-4eb6-8c43-c97da52e6b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.459119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd94891-eb35-4eb6-8c43-c97da52e6b4b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.459488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd94891-eb35-4eb6-8c43-c97da52e6b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.459693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd94891-eb35-4eb6-8c43-c97da52e6b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.469840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd94891-eb35-4eb6-8c43-c97da52e6b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:21.471533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd94891-eb35-4eb6-8c43-c97da52e6b4b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:21.482572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1023/2053] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:21.646795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:21.699903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:21.716413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e84cda-4624-47fd-93de-19aa13272768 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.721374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.722252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.722573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.722876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] PATH_INFO: `/v3/users/0ac1e691aa63436e9e74de615f653749` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.724420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac1e691aa63436e9e74de615f653749)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:21.727115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:21.780068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac1e691aa63436e9e74de615f653749'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:21.780730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac1e691aa63436e9e74de615f653749'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:21.788044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac1e691aa63436e9e74de615f653749'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:21.788684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac1e691aa63436e9e74de615f653749'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:21.796929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac1e691aa63436e9e74de615f653749'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:21.814285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1031/2054] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:21 2022] DELETE /identity/v3/users/0ac1e691aa63436e9e74de615f653749 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:21.819212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.819672 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.819763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:21.838705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:21.839926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:21.901117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1024/2055] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:21.910477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:21.912149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:21.986151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56645e7-62af-4bc9-8542-962977815598 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:21.989996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:21.990760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:21.991183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:21.991532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:22.002749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:22.004560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:22.030795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1032/2056] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:22.453046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:22.454670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:22.469039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fddc9-6519-479e-9f95-fd93094b9389 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:22.489914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:22.491160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:22.491528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:22.491861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] PATH_INFO: `/v3/projects/89f823d212244767a0b83e6779269cf6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:22.493254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=89f823d212244767a0b83e6779269cf6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:22.496212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:22.583311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89f823d212244767a0b83e6779269cf6', 'request_id': 'req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.589835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89f823d212244767a0b83e6779269cf6', 'request_id': 'req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.597350 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89f823d212244767a0b83e6779269cf6', 'request_id': 'req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.625753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] The token cache is being invalidate because project 89f823d212244767a0b83e6779269cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:22.626134 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.626926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.628840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1025/2057] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:22 2022] DELETE /identity/v3/projects/89f823d212244767a0b83e6779269cf6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:22.632607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:22.650922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd22ad0-d309-41e4-be32-82a7417dd202 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:22.723337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:22.724216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:22.724555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:22.724853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] PATH_INFO: `/v3/users/b07741297e4e462396a217abc2a3e4d9` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:22.726265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07741297e4e462396a217abc2a3e4d9)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:22.729711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:22.745658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:22.788649 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07741297e4e462396a217abc2a3e4d9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.789310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07741297e4e462396a217abc2a3e4d9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.797164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07741297e4e462396a217abc2a3e4d9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.797853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07741297e4e462396a217abc2a3e4d9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.806316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5bf6b948-5d6c-4195-a7f0-5083901cefa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07741297e4e462396a217abc2a3e4d9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:22.835499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1033/2058] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:22 2022] DELETE /identity/v3/users/b07741297e4e462396a217abc2a3e4d9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:23.140284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.142201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:23.156371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd3166-5f3b-410e-940a-00ea2af5b3ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.209655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:23.228931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.230825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.231352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.231736 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] PATH_INFO: `/v3/projects/77ad7d78cb9848339d1c47142b0eb696` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.233314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ad7d78cb9848339d1c47142b0eb696)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.236109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.297884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ad7d78cb9848339d1c47142b0eb696', 'request_id': 'req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:23.304101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ad7d78cb9848339d1c47142b0eb696', 'request_id': 'req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:23.311617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ad7d78cb9848339d1c47142b0eb696', 'request_id': 'req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:23.326923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] The token cache is being invalidate because project 77ad7d78cb9848339d1c47142b0eb696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:23.327145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:23.328044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-658866f6-c88e-4a64-94dc-dcf7bb5ce44a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:23.329592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1026/2059] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:23 2022] DELETE /identity/v3/projects/77ad7d78cb9848339d1c47142b0eb696 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:23.334261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.335482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.335780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.356969 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:23.358295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:23.429261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1034/2060] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:23.438401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.438695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.439043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.457966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:23.459239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:23.467949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1027/2061] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:23.472156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.483324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725e364-77a0-4654-9db4-b46fe9ca16f9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.487814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.488472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.488757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.489034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.489758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:05:23.490075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.491720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.504717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1035/2062] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:05:23.508932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.518239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaca50a-41d5-4924-bfe5-1679df6c6bef None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.523537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.524203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.524533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.524767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.525524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.527469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.554243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1028/2063] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:23 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:23.563740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.575695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a69cef-bc33-445b-b508-75e8be5bd73b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.640218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.640988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.642607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.642998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.643821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.645611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.668805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1036/2064] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:23 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:23.672589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.696099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa313ba-7e0d-4ace-aa64-840299b4b5bc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.703490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.704477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.705079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.706087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.707436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.709862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.724964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1029/2065] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:23.728822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.738749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f68-18c6-40ee-9a81-9630c0c9c272 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.744492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f368750-ef47-4c0c-be76-717d129a2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.744742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f368750-ef47-4c0c-be76-717d129a2e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.745215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f368750-ef47-4c0c-be76-717d129a2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.745578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f368750-ef47-4c0c-be76-717d129a2e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.746463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f368750-ef47-4c0c-be76-717d129a2e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.748121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f368750-ef47-4c0c-be76-717d129a2e1f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.749671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:23.761800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1037/2066] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:23.765821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:23.776860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfe1c1-80ac-4888-9ff4-9e2a7d8ea5cb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:23.783724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:23.784916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.785346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.786055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] PATH_INFO: `/v3/projects/a2541674a0a74564b5c63543e5a674b8/users/6335f13497c6427fa41c82a9d405c953/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.826983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2541674a0a74564b5c63543e5a674b8, user_id=6335f13497c6427fa41c82a9d405c953, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:23.836498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:23.852519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1030/2067] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:23 2022] PUT /identity/v3/projects/a2541674a0a74564b5c63543e5a674b8/users/6335f13497c6427fa41c82a9d405c953/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:23.865497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:23.865808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:23.866063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:23.895569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] MFA Rules not processed for user `6335f13497c6427fa41c82a9d405c953`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:23.896838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:23.966028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1038/2068] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:23.971657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.040325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850ba8c-013d-462e-af29-1aab7ec7a137 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.045968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.048581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.048581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.048581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.048581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.049773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.076802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1031/2069] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:23 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:24.080918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.093059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7793c5-9f68-4e34-b798-1527c5ca64e4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.156389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.157086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.157367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.157657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.158386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.160524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.185478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1039/2070] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:24 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:24.186718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.192172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:24.198258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923cee6-0d08-45c8-8f1e-1638ad7562fe admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.203482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.204209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.204568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.205053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.205986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.207940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.223642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1032/2071] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:05:24.224561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.234944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaebbb-b1e0-4696-b92b-c006d0392f64 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.239677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a2b26-dcb6-498e-ab52-7e9d918999c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.240345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a2b26-dcb6-498e-ab52-7e9d918999c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.240634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a2b26-dcb6-498e-ab52-7e9d918999c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.240916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a2b26-dcb6-498e-ab52-7e9d918999c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.241664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a2b26-dcb6-498e-ab52-7e9d918999c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.243331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a2b26-dcb6-498e-ab52-7e9d918999c6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.253873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1040/2072] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:05:24.258340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.269554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574d2bf-bae1-4f44-ae87-4ae858d77e1a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.274231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.275814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.276098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.276377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] PATH_INFO: `/v3/projects/823c0c1c79f44f97941754e89ae336d4/users/f20217e993314f4dbac668830297ace2/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.291262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=823c0c1c79f44f97941754e89ae336d4, user_id=f20217e993314f4dbac668830297ace2, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.293971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.308026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1033/2073] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:24 2022] PUT /identity/v3/projects/823c0c1c79f44f97941754e89ae336d4/users/f20217e993314f4dbac668830297ace2/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:24.319985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.320284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.320571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.349822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] MFA Rules not processed for user `f20217e993314f4dbac668830297ace2`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:24.351055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:24.411560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:24.422563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1041/2074] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:24.427109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.504708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74725e67-745f-4943-995f-24b5339bee11 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.510190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.511175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.511468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.511793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.512500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.514432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.542496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1034/2075] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:24 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:24.546591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.558787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8034e8b-4cce-441e-9e0c-4e87ab9e6c6b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.618447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.619750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.620032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.620312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.620993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.622959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.643753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1042/2076] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:24 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:24.647697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.658290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1bac2-0ac8-4053-b57e-0a0e67c6e5b4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.663172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.663832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.664122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.664405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.665126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.666734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.678287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1035/2077] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:24.683099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.693608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a8e1e-b97d-43af-9105-ae27fa70043b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.700475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.702034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.702556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.703107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] PATH_INFO: `/v3/projects/8c490b78150341a4829a3973b2d1e649/users/809d27d844774cbebc3f6f05cbc63920/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.721075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c490b78150341a4829a3973b2d1e649, user_id=809d27d844774cbebc3f6f05cbc63920, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.723818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.741668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1043/2078] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:24 2022] PUT /identity/v3/projects/8c490b78150341a4829a3973b2d1e649/users/809d27d844774cbebc3f6f05cbc63920/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:24.742432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.757983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f8670-7414-49cf-9a1a-057862d59e60 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.825372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7a5d3-e34b-4e35-a601-d38696244640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.826045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7a5d3-e34b-4e35-a601-d38696244640 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.826330 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7a5d3-e34b-4e35-a601-d38696244640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.826611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7a5d3-e34b-4e35-a601-d38696244640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.827450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7a5d3-e34b-4e35-a601-d38696244640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.829080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7a5d3-e34b-4e35-a601-d38696244640 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.839591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1036/2079] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:24.844050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:24.854869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f447b03-aa7d-42c2-81f5-a6081612ef4d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:24.859841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:24.860732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.861045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.861324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] PATH_INFO: `/v3/projects/8c490b78150341a4829a3973b2d1e649/users/809d27d844774cbebc3f6f05cbc63920/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.863367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c490b78150341a4829a3973b2d1e649, user_id=809d27d844774cbebc3f6f05cbc63920, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:24.866043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:24.876794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1044/2080] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:24 2022] PUT /identity/v3/projects/8c490b78150341a4829a3973b2d1e649/users/809d27d844774cbebc3f6f05cbc63920/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:24.890711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:24.891054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:24.891366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:24.923884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] MFA Rules not processed for user `809d27d844774cbebc3f6f05cbc63920`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:24.925083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:24.955429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:25.001588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1037/2081] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:25.012735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:25.014574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:25.075738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d94d7e-db31-4692-bf57-ede3584031e9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:25.079390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:25.080031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:25.080315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:25.080592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:25.089816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:25.091307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:25.101161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1045/2082] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:05:25.595045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:25.596479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:25.608825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647b15c-c07e-4683-acfc-32d205e831c4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:25.613053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:25.613701 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:25.613991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:25.614279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:25.672417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:25.673855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:25.683047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1038/2083] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:30.814003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:30.823957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161b5c-d37f-4dec-86e6-ddba0aa056b7 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:30.827645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8586fd-2da5-4627-a722-30cbd0dc11b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:30.828284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8586fd-2da5-4627-a722-30cbd0dc11b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:30.828565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8586fd-2da5-4627-a722-30cbd0dc11b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:30.828840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8586fd-2da5-4627-a722-30cbd0dc11b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:30.876956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:30.893228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8586fd-2da5-4627-a722-30cbd0dc11b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:30.894708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8586fd-2da5-4627-a722-30cbd0dc11b0 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:30.903980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1046/2084] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:33.049332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:33.111640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba8d1c-8c9d-496a-a534-9c997346715b service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:33.116043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:33.116894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:33.117176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:33.117453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] PATH_INFO: `/v3/users/6335f13497c6427fa41c82a9d405c953` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:33.118661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6335f13497c6427fa41c82a9d405c953)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:33.120752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:33.175781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6335f13497c6427fa41c82a9d405c953'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.176334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6335f13497c6427fa41c82a9d405c953'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.177044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6335f13497c6427fa41c82a9d405c953'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.186318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6335f13497c6427fa41c82a9d405c953'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.197617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:33.208398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6335f13497c6427fa41c82a9d405c953'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.223118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1039/2085] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:33 2022] DELETE /identity/v3/users/6335f13497c6427fa41c82a9d405c953 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:33.227832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:33.228563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:33.228563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:33.245641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:33.246892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:33.308518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1047/2086] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:33.317762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:33.319203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:33.376845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3095fcd-fda6-461d-a104-8d39c6c046b2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:33.380430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:33.381386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:33.381677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:33.381957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:33.391459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:33.392881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:33.402636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1040/2087] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:33.666083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:33.667543 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:33.684225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a7d70-f51b-4d57-ab9e-254eb92610dc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:33.689234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:33.690284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:33.690479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:33.690758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] PATH_INFO: `/v3/projects/a2541674a0a74564b5c63543e5a674b8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:33.692125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2541674a0a74564b5c63543e5a674b8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:33.694074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:33.741361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2541674a0a74564b5c63543e5a674b8', 'request_id': 'req-eecad576-c687-4153-aa6c-e4536b6ebde7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.748255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2541674a0a74564b5c63543e5a674b8', 'request_id': 'req-eecad576-c687-4153-aa6c-e4536b6ebde7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.755538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2541674a0a74564b5c63543e5a674b8', 'request_id': 'req-eecad576-c687-4153-aa6c-e4536b6ebde7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.766482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] The token cache is being invalidate because project a2541674a0a74564b5c63543e5a674b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:33.766777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.767434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.768974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1048/2088] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:33 2022] DELETE /identity/v3/projects/a2541674a0a74564b5c63543e5a674b8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:33.773203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:33.782700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f647-f0de-40a8-80d0-5ee688e4aefd service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:33.840564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:33.841438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:33.841734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:33.842015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] PATH_INFO: `/v3/users/f20217e993314f4dbac668830297ace2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:33.843488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20217e993314f4dbac668830297ace2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:33.845451 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:33.893427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20217e993314f4dbac668830297ace2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.894115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20217e993314f4dbac668830297ace2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.894727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20217e993314f4dbac668830297ace2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.901824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20217e993314f4dbac668830297ace2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.915463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20217e993314f4dbac668830297ace2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:33.930749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1041/2089] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:33 2022] DELETE /identity/v3/users/f20217e993314f4dbac668830297ace2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:34.226628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:34.230190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:34.253768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecad576-c687-4153-aa6c-e4536b6ebde7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:34.315406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:34.316276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:34.316565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:34.316880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] PATH_INFO: `/v3/projects/823c0c1c79f44f97941754e89ae336d4` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:34.318067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] RBAC: Authorizing `identity:delete_project(project_id=823c0c1c79f44f97941754e89ae336d4)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:34.320048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:34.332001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:34.372798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823c0c1c79f44f97941754e89ae336d4', 'request_id': 'req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.379282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823c0c1c79f44f97941754e89ae336d4', 'request_id': 'req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.388014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823c0c1c79f44f97941754e89ae336d4', 'request_id': 'req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.398772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] The token cache is being invalidate because project 823c0c1c79f44f97941754e89ae336d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:34.399090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.399726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.401869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1049/2090] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:34 2022] DELETE /identity/v3/projects/823c0c1c79f44f97941754e89ae336d4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:34.405887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:34.407359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:34.415629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:34.423098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca1bda-930f-496c-9648-78daa2d076be admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:34.490799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:34.491804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:34.492119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:34.492400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] PATH_INFO: `/v3/users/809d27d844774cbebc3f6f05cbc63920` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:34.493625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] RBAC: Authorizing `identity:delete_user(user_id=809d27d844774cbebc3f6f05cbc63920)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:34.495810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:34.543841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809d27d844774cbebc3f6f05cbc63920'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.544550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809d27d844774cbebc3f6f05cbc63920'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.545155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809d27d844774cbebc3f6f05cbc63920'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.552920 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809d27d844774cbebc3f6f05cbc63920'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.564577 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d137be51-23f7-47b3-b430-ddeae034904a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809d27d844774cbebc3f6f05cbc63920'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.577197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1042/2091] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:34 2022] DELETE /identity/v3/users/809d27d844774cbebc3f6f05cbc63920 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:05:34.853178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:34.854705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:34.867370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1b8f8-d8c2-4638-910a-f97cd058e24d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:34.924415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:34.925271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:34.925557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:34.925834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] PATH_INFO: `/v3/projects/8c490b78150341a4829a3973b2d1e649` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:34.927036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c490b78150341a4829a3973b2d1e649)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:34.928965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:34.974524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c490b78150341a4829a3973b2d1e649', 'request_id': 'req-7081f50f-fe89-4c4c-919f-f41678a0eab6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.980807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c490b78150341a4829a3973b2d1e649', 'request_id': 'req-7081f50f-fe89-4c4c-919f-f41678a0eab6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.987695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c490b78150341a4829a3973b2d1e649', 'request_id': 'req-7081f50f-fe89-4c4c-919f-f41678a0eab6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.998605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] The token cache is being invalidate because project 8c490b78150341a4829a3973b2d1e649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:34.998927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:34.999545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7081f50f-fe89-4c4c-919f-f41678a0eab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:35.001074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1050/2092] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:34 2022] DELETE /identity/v3/projects/8c490b78150341a4829a3973b2d1e649 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:35.008025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.008322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.008600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.027392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:35.028584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:35.088993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1043/2093] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:35.097536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.097905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.098255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.116658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:35.117897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:35.126490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1051/2094] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:35.130671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.141170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840cbd-fda8-47f0-8155-0cb456d8d80a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.147980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.148986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.149287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.149580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.150406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:05:35.150795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.152473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.164654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1044/2095] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:05:35.169007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.178287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e5f1-f3da-4de3-bb97-93612265b038 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.183482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.184141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.184426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.184704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.185409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.187373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.217089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1052/2096] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:35.218580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.228994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0059d-397e-4f1e-9078-ef101ee58178 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.233770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:35.290811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.291908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.292427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.292952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.295093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.297565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.318418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1045/2097] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:35.323191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.328505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:35.334675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf17c8-53bf-4273-abe5-3cd48a7efaac admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.340913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.341548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.341958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.342209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.342976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.344861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.355939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1053/2098] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:35.359960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.370578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760df01a-2c17-49f9-8a20-b94afaa364a3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.375849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ab64d-d7af-45d6-8184-57078758355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.376508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ab64d-d7af-45d6-8184-57078758355e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.376800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ab64d-d7af-45d6-8184-57078758355e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.377085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ab64d-d7af-45d6-8184-57078758355e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.377849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ab64d-d7af-45d6-8184-57078758355e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.379531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ab64d-d7af-45d6-8184-57078758355e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.391657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1046/2099] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:35.396617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.408841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610de6d-cb21-4e0c-be39-25123ab334e5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.413570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.414482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.414768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.415089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] PATH_INFO: `/v3/projects/dfa9233db47f405ab27631cfaa33d08a/users/db7960e855b74c638dcc559349c169a3/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.430785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa9233db47f405ab27631cfaa33d08a, user_id=db7960e855b74c638dcc559349c169a3, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.434173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.447039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1054/2100] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:35 2022] PUT /identity/v3/projects/dfa9233db47f405ab27631cfaa33d08a/users/db7960e855b74c638dcc559349c169a3/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:35.458597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.459107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.459221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.487225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] MFA Rules not processed for user `db7960e855b74c638dcc559349c169a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:35.488450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:35.559615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1047/2101] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:35.564442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.650358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70530622-1b54-4c67-bf4c-c2566587875a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.650358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.650358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.650358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.650833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.650833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.651428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.682109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1055/2102] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:05:35.683388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.694459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7d9f-5e50-4cd1-9317-6ea4d6c02bec None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.754824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.755524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.755816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.756113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.756781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.758694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.779788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1048/2103] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:35 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:35.783844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.794942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef824b29-149a-4ed6-b07d-af94c98eaa36 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.799538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.800203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.800492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.800776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.801496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.803160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.814135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1056/2104] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:35.818480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.829246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087696c-ba37-4f3c-a598-a19af795fea3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.833866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.834746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.835078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.835362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] PATH_INFO: `/v3/projects/cebd50adacea4805ba1de88b1d87ac63/users/8ef896b4eacb4aaeab44f176f73a30be/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.840760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:35.858342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebd50adacea4805ba1de88b1d87ac63, user_id=8ef896b4eacb4aaeab44f176f73a30be, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.861147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.875265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1049/2105] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:35 2022] PUT /identity/v3/projects/cebd50adacea4805ba1de88b1d87ac63/users/8ef896b4eacb4aaeab44f176f73a30be/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:35.879256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.891048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97431c13-6808-4f21-81c1-5754d2f799cf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.950183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ceb90-a8dd-4148-87ce-2226ba23c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.950903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ceb90-a8dd-4148-87ce-2226ba23c89d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.951749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ceb90-a8dd-4148-87ce-2226ba23c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.951749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ceb90-a8dd-4148-87ce-2226ba23c89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.952414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ceb90-a8dd-4148-87ce-2226ba23c89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.953762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ceb90-a8dd-4148-87ce-2226ba23c89d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:35.965228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1057/2106] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:35.969556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:35.980180 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b445-20d5-425d-8d2a-5cfb6182d96f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:35.984784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:35.985655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:35.985938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:35.986222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] PATH_INFO: `/v3/projects/cebd50adacea4805ba1de88b1d87ac63/users/8ef896b4eacb4aaeab44f176f73a30be/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:35.988165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebd50adacea4805ba1de88b1d87ac63, user_id=8ef896b4eacb4aaeab44f176f73a30be, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:35.990798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:36.002570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1050/2107] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:35 2022] PUT /identity/v3/projects/cebd50adacea4805ba1de88b1d87ac63/users/8ef896b4eacb4aaeab44f176f73a30be/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:05:36.015443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:36.015443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:36.015443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:36.043492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] MFA Rules not processed for user `8ef896b4eacb4aaeab44f176f73a30be`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:36.044772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:36.075319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:36.130740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1058/2108] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:36 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:36.141464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:36.143241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:36.205918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c04c-438c-4552-b3ff-90496e5b9765 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:36.210301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:36.211007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:36.211322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:36.211609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:36.221720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:36.223234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:36.233939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1051/2109] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:36.381241 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:36.383088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:36.395660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff3f24-f620-49fc-98d0-7cbbc8e95968 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:36.399568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742878e6-8a59-4225-9b63-5cf6cfb05907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:36.400849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742878e6-8a59-4225-9b63-5cf6cfb05907 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:36.401160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742878e6-8a59-4225-9b63-5cf6cfb05907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:36.401471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742878e6-8a59-4225-9b63-5cf6cfb05907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:36.460921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742878e6-8a59-4225-9b63-5cf6cfb05907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:36.462386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742878e6-8a59-4225-9b63-5cf6cfb05907 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:36.471756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1059/2110] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:46.229493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=38,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:46.467482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=45 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:49.762192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:49.763643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:49.828896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ad863-5aec-4540-aa9b-af33a0ea6ae6 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:49.833313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:49.834178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:49.834462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:49.834739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] PATH_INFO: `/v3/users/aaf45acacace44eba6d9e73cfdfe4a28` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:49.835949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf45acacace44eba6d9e73cfdfe4a28)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:49.837885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:49.903078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf45acacace44eba6d9e73cfdfe4a28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:49.903817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf45acacace44eba6d9e73cfdfe4a28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:49.904570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf45acacace44eba6d9e73cfdfe4a28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:49.914478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf45acacace44eba6d9e73cfdfe4a28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:49.929261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a4b7bd73-bbab-41ab-beb3-1d40082f43a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf45acacace44eba6d9e73cfdfe4a28'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:49.948102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1052/2111] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:49 2022] DELETE /identity/v3/users/aaf45acacace44eba6d9e73cfdfe4a28 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:49.952125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:49.952596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:49.952712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:49.952882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:49.952976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:49.953218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:49.972886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:49.974691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:49.974803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:49.976094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.036410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1060/2112] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:50.048385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1053/2113] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:50.058986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:50.060622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.089974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:50.091624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.117457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:50.131788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e97ee8-ac53-4b30-b8d6-17a65a51324f None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:50.136784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:50.137855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.138144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.138513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.149606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:50.151107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:50.162505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1061/2114] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:50.185716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ce169-a8f5-425b-8d5d-8ab35b2e14f4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:50.191329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c170a-46e4-4339-a0c0-80863d039e95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:50.192045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c170a-46e4-4339-a0c0-80863d039e95 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.192342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c170a-46e4-4339-a0c0-80863d039e95 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.192617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c170a-46e4-4339-a0c0-80863d039e95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.202555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c170a-46e4-4339-a0c0-80863d039e95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:50.204339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c170a-46e4-4339-a0c0-80863d039e95 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:50.206632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:50.208193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.214195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1054/2115] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 12:05:50.223235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2b512-4789-4f02-8ba6-c7546008f86c service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:50.229351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:50.230437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.230807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.231651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] PATH_INFO: `/v3/users/db7960e855b74c638dcc559349c169a3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.233594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7960e855b74c638dcc559349c169a3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:50.235540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:50.328097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7960e855b74c638dcc559349c169a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.328866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7960e855b74c638dcc559349c169a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.345279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7960e855b74c638dcc559349c169a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.346112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7960e855b74c638dcc559349c169a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.359492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7960e855b74c638dcc559349c169a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.401520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1062/2116] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:50 2022] DELETE /identity/v3/users/db7960e855b74c638dcc559349c169a3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:05:50.406346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.406952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.407418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.430465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:50.432057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.511346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1055/2117] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:50.522205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:50.524122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.564737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:50.628812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83602cd4-d8e1-48c8-80c9-5b7107d3acce admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:50.635078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb084e7-982c-4b0f-ad31-f2f56031e319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:50.635990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb084e7-982c-4b0f-ad31-f2f56031e319 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.637638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb084e7-982c-4b0f-ad31-f2f56031e319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.637953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb084e7-982c-4b0f-ad31-f2f56031e319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.651557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb084e7-982c-4b0f-ad31-f2f56031e319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:50.654574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb084e7-982c-4b0f-ad31-f2f56031e319 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:50.672088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1063/2118] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:05:50.736867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:50.739220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:50.765753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e9b53-75db-42e2-86fd-f781a12f4d58 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:50.778968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:50.779992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.780570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.781374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] PATH_INFO: `/v3/projects/a1b303a5b7f24930acdd0b05d6ed8e91` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.783442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b303a5b7f24930acdd0b05d6ed8e91)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:50.786676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:50.849957 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b303a5b7f24930acdd0b05d6ed8e91', 'request_id': 'req-85a1af4c-47a7-431a-868e-3c50f859147c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.857579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b303a5b7f24930acdd0b05d6ed8e91', 'request_id': 'req-85a1af4c-47a7-431a-868e-3c50f859147c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.865063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b303a5b7f24930acdd0b05d6ed8e91', 'request_id': 'req-85a1af4c-47a7-431a-868e-3c50f859147c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.903465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] The token cache is being invalidate because project a1b303a5b7f24930acdd0b05d6ed8e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:50.904817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.906077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-85a1af4c-47a7-431a-868e-3c50f859147c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:50.911488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1056/2119] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:50 2022] DELETE /identity/v3/projects/a1b303a5b7f24930acdd0b05d6ed8e91 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:50.916343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:50.917021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:50.919022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:50.943903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:50.945275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.036671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1064/2120] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:51.045296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.045773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.046391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.096867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:51.098389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.168906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1057/2121] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:51.169716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.198140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1d50d-130c-42c9-b643-96c2cce2dff3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.215920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.218112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.218650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.219546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.227901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:05:51.227901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.227901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.246165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1065/2122] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:05:51.253353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.263240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a6fa-dec7-486b-8716-96c6f2fba347 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.268840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.269850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.270417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.271066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.272467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.275625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.310419 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.314197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.340609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1058/2123] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:51 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:51.345051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26090af1-1e3a-4311-a922-bbf5895ca804 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.346862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.358283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:51.446054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238751b-2115-4f26-a6a8-c96e26d710a0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.450938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.451638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.451954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.452225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.452916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.455102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.474081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.476128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.476501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.477601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] PATH_INFO: `/v3/projects/dfa9233db47f405ab27631cfaa33d08a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.478980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa9233db47f405ab27631cfaa33d08a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.481474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.483862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1059/2124] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:51 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:51.487879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.503708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884acbdc-7f87-4eaf-a639-b326b4d2bded admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.527939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.528740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.529366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.529887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.530705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.532415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.549176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1060/2125] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:51.552961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.564869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa9233db47f405ab27631cfaa33d08a', 'request_id': 'req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.567249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:51.569997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99ed72-7581-401c-b319-3863ed326e6a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.572741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa9233db47f405ab27631cfaa33d08a', 'request_id': 'req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.584027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa9233db47f405ab27631cfaa33d08a', 'request_id': 'req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.600956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] The token cache is being invalidate because project dfa9233db47f405ab27631cfaa33d08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:51.601346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.602107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.604011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1066/2126] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:51 2022] DELETE /identity/v3/projects/dfa9233db47f405ab27631cfaa33d08a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:51.608614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.614231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.651586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.653804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.654280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.654733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.655515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.657165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.674142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1061/2127] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:51.674991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.678008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.700576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696be1d-5a5a-4503-8984-111d95bafbcf admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.708650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c017f-7efe-4077-8270-37c53d8c7357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.710075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c017f-7efe-4077-8270-37c53d8c7357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.710439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c017f-7efe-4077-8270-37c53d8c7357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.710853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c017f-7efe-4077-8270-37c53d8c7357 admin admin] PATH_INFO: `/v3/projects/a6725c8fde0e4bce93716183772e5dbd/users/601038b478b6408ab4a8e44ffc2d6cb7/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.735475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c017f-7efe-4077-8270-37c53d8c7357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6725c8fde0e4bce93716183772e5dbd, user_id=601038b478b6408ab4a8e44ffc2d6cb7, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.738595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c017f-7efe-4077-8270-37c53d8c7357 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.752950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f23630-cbf3-478e-9e4f-6cd7fdda85ab admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:51.764737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1062/2128] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:51 2022] PUT /identity/v3/projects/a6725c8fde0e4bce93716183772e5dbd/users/601038b478b6408ab4a8e44ffc2d6cb7/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:51.775613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.776480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.776480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.840164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] MFA Rules not processed for user `601038b478b6408ab4a8e44ffc2d6cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:51.841547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.842560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:51.843629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:51.843908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:51.844549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] PATH_INFO: `/v3/users/8ef896b4eacb4aaeab44f176f73a30be` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:51.846102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef896b4eacb4aaeab44f176f73a30be)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:51.848391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:51.910748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef896b4eacb4aaeab44f176f73a30be'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.911273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:51.911867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef896b4eacb4aaeab44f176f73a30be'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.943700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef896b4eacb4aaeab44f176f73a30be'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.946153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef896b4eacb4aaeab44f176f73a30be'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.955913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef896b4eacb4aaeab44f176f73a30be'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:51.958058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1063/2129] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:51.973388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:51.974999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:51.992156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1067/2130] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:05:51 2022] DELETE /identity/v3/users/8ef896b4eacb4aaeab44f176f73a30be => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:52.077653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8090d5-0d41-44ff-a849-14071d55886b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:52.154495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:52.155231 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:52.155520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:52.156010 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:52.157855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:52.271177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:52.273135 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa715b1-27a3-47b9-b5a9-3421fd7e099d service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:52.311056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1064/2131] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:05:52.439239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:52.441072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:52.459365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:52.617922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a6755-040e-4d06-833d-6aacd8b112df admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:52.623762 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:52.624771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:52.625267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:52.625645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] PATH_INFO: `/v3/projects/cebd50adacea4805ba1de88b1d87ac63` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:52.627021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebd50adacea4805ba1de88b1d87ac63)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:52.629813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:52.701752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebd50adacea4805ba1de88b1d87ac63', 'request_id': 'req-5605e082-2218-443a-ae24-84179fbd9ed2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:52.718150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebd50adacea4805ba1de88b1d87ac63', 'request_id': 'req-5605e082-2218-443a-ae24-84179fbd9ed2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:52.729883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebd50adacea4805ba1de88b1d87ac63', 'request_id': 'req-5605e082-2218-443a-ae24-84179fbd9ed2'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:52.751372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] The token cache is being invalidate because project cebd50adacea4805ba1de88b1d87ac63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:05:52.751854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:52.752717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-5605e082-2218-443a-ae24-84179fbd9ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:05:52.754814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1068/2132] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:05:52 2022] DELETE /identity/v3/projects/cebd50adacea4805ba1de88b1d87ac63 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:52.759547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:52.760614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:52.761583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:52.820156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:52.821546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:52.976624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:52.991810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1065/2133] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:53.000996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.001883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.002193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.037124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:53.038512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:53.053112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.060941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1069/2134] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:53.065460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a15831-987c-4e51-abe0-fc2f4e852d98 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.075130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.076240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.076728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.077219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.078141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:05:53.078616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.080427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.095074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1066/2135] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:05:53.099412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.109040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dddc3-c63b-4d28-b20e-b808deb93d70 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.113690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.114356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.114641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.115029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.115734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.117818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.147399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1070/2136] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:53 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:53.151291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.166304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26cdc9-4110-4f1f-b928-d43beab5c091 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.271872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.273073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.273442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.273723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.274455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.276683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.324768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1067/2137] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:53 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:53.327420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.359523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcdbad-6299-4404-9f30-d5e8718e2d70 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.368710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.369722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.370191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.371085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.373981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.376640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.399587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1071/2138] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:53.401173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.414102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc6dd1-3b9e-45f2-b3b2-f715822338f3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.420351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c42014-e262-439a-9497-9a56633338fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.421278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c42014-e262-439a-9497-9a56633338fc admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.421579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c42014-e262-439a-9497-9a56633338fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.421982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c42014-e262-439a-9497-9a56633338fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.422728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c42014-e262-439a-9497-9a56633338fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.424390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c42014-e262-439a-9497-9a56633338fc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.437480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1068/2139] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:53.441387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.453097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526d3e2-a94b-4129-9c4f-d02ae93d885d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.458295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.459457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.459748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.460041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] PATH_INFO: `/v3/projects/9732055c72d44d1da18d13a40a9fc8e5/users/52d4af4adb43404c951e5b395a5332d3/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.476982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9732055c72d44d1da18d13a40a9fc8e5, user_id=52d4af4adb43404c951e5b395a5332d3, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.479591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.498028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1072/2140] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:53 2022] PUT /identity/v3/projects/9732055c72d44d1da18d13a40a9fc8e5/users/52d4af4adb43404c951e5b395a5332d3/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:05:53.510876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.510876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.510876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.538611 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] MFA Rules not processed for user `52d4af4adb43404c951e5b395a5332d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:53.539890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:53.634085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1069/2141] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:53.638655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.677254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:53.711182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161398-c1fc-4347-82bf-034c4ab5c212 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.716933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7b021-e403-4251-9ee5-9f4defedec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.717627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7b021-e403-4251-9ee5-9f4defedec0e admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.717920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7b021-e403-4251-9ee5-9f4defedec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.718207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7b021-e403-4251-9ee5-9f4defedec0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.718941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7b021-e403-4251-9ee5-9f4defedec0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.721184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7b021-e403-4251-9ee5-9f4defedec0e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.760741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1073/2142] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:05:53 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:53.762423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.777573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee5ffe-c1d2-4790-8efa-660413969837 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.852170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:53.883336 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.884491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.884752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.885022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.885938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.891120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.906749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.907111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.907371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.923217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1070/2143] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:05:53 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:05:53.927157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:53.929943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] MFA Rules not processed for user `601038b478b6408ab4a8e44ffc2d6cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:53.931338 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:53.938753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3fceb-eb8d-4d9b-beab-2700c22a47d3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:53.955707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:53.956787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:53.957079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:53.957361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:53.958098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:53.959809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:53.999506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1071/2144] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:05:54.007717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:54.026267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f4e73-0b28-4c42-92bb-b2f39238c934 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:54.026503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1074/2145] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:05:54.056771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:54.057610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:54.057896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:54.058218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] PATH_INFO: `/v3/projects/e38a7363d9cf4257bca212e579d80542/users/fa672cde53354fc58003e97201fdd6b3/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:54.089130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38a7363d9cf4257bca212e579d80542, user_id=fa672cde53354fc58003e97201fdd6b3, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:54.102824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:54.127227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1072/2146] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:54 2022] PUT /identity/v3/projects/e38a7363d9cf4257bca212e579d80542/users/fa672cde53354fc58003e97201fdd6b3/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:54.128511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:54.144282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9386be6-3ec3-4312-a9bb-fda4f0947a1c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:54.226456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa738c-c915-418d-a488-b0e24ccb4997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:54.227331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa738c-c915-418d-a488-b0e24ccb4997 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:54.227723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa738c-c915-418d-a488-b0e24ccb4997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:54.228108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa738c-c915-418d-a488-b0e24ccb4997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:54.228944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa738c-c915-418d-a488-b0e24ccb4997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:54.231028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa738c-c915-418d-a488-b0e24ccb4997 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:54.242542 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1075/2147] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:05:54.246534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:54.264888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e350121-2d34-4956-8da9-3b214ac367af admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:54.274563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:54.275976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:54.276267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:54.276551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] PATH_INFO: `/v3/projects/e38a7363d9cf4257bca212e579d80542/users/fa672cde53354fc58003e97201fdd6b3/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:54.278682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38a7363d9cf4257bca212e579d80542, user_id=fa672cde53354fc58003e97201fdd6b3, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:54.281389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:54.293566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1073/2148] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:05:54 2022] PUT /identity/v3/projects/e38a7363d9cf4257bca212e579d80542/users/fa672cde53354fc58003e97201fdd6b3/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:05:54.304822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:54.305390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:54.305826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:54.340706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] MFA Rules not processed for user `fa672cde53354fc58003e97201fdd6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:05:54.342385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:54.438334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:05:54.475152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1076/2149] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:05:54.488108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:05:54.489933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:05:54.575860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d85fd-2f81-47a9-862d-a4d9a12d15db admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:05:54.580337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d9985-4bce-47f4-9ca3-501d530a976e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:05:54.581209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d9985-4bce-47f4-9ca3-501d530a976e service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:05:54.581571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d9985-4bce-47f4-9ca3-501d530a976e service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:05:54.581927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d9985-4bce-47f4-9ca3-501d530a976e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:05:54.645648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d9985-4bce-47f4-9ca3-501d530a976e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:05:54.647768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d9985-4bce-47f4-9ca3-501d530a976e service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:05:54.658757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1074/2150] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:05:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:00.566092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:00.568372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:00.644172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481a561-9d17-4127-95e9-c8a05fac8b09 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:00.648367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:00.649051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:00.649343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:00.649631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:00.746900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:00.748390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:00.781218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1077/2151] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:06:01.886943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76d0c2-d383-4da4-ac23-6ffaaee86f96 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:01.888293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76d0c2-d383-4da4-ac23-6ffaaee86f96 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:01.888882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76d0c2-d383-4da4-ac23-6ffaaee86f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:01.914509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3f76d0c2-d383-4da4-ac23-6ffaaee86f96 None None] MFA Rules not processed for user `473532c21cad4c3492c8425855993b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:01.916345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3f76d0c2-d383-4da4-ac23-6ffaaee86f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:01.966146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:01.982365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1075/2152] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:06.194528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:06.435908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561aad5-35dd-45cb-a3a7-34f650b70273 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:06.441993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:06.442924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:06.443221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:06.443509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] PATH_INFO: `/v3/users/52d4af4adb43404c951e5b395a5332d3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:06.444881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d4af4adb43404c951e5b395a5332d3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:06.446881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:06.542627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d4af4adb43404c951e5b395a5332d3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:06.543863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d4af4adb43404c951e5b395a5332d3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:06.584079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d4af4adb43404c951e5b395a5332d3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:06.587287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d4af4adb43404c951e5b395a5332d3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:06.600018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d4af4adb43404c951e5b395a5332d3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:06.607805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:06.639664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1078/2153] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:06:06 2022] DELETE /identity/v3/users/52d4af4adb43404c951e5b395a5332d3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:06.641536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:06.644087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:06.644392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:06.699362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:06.700753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:06.791232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1076/2154] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:06.801275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:06.803047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:06.912231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e40c2-273b-4cf7-9a33-ba45bea1973e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:06.915966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:06.916715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:06.917075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:06.917433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:06.934632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:06.936324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:06.962450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1079/2155] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:07.261235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:07.263603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:07.276393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622c53-55f8-41f5-83f6-0b7807080309 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:07.281634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:07.282513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:07.282803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:07.283122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] PATH_INFO: `/v3/projects/9732055c72d44d1da18d13a40a9fc8e5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:07.284318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9732055c72d44d1da18d13a40a9fc8e5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:07.286359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:07.345369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9732055c72d44d1da18d13a40a9fc8e5', 'request_id': 'req-4f71a302-4721-4d65-a7c5-f77dce3cdba1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.352596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9732055c72d44d1da18d13a40a9fc8e5', 'request_id': 'req-4f71a302-4721-4d65-a7c5-f77dce3cdba1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.362306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9732055c72d44d1da18d13a40a9fc8e5', 'request_id': 'req-4f71a302-4721-4d65-a7c5-f77dce3cdba1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.380409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] The token cache is being invalidate because project 9732055c72d44d1da18d13a40a9fc8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:06:07.380705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.381394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.385978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1077/2156] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:06:07 2022] DELETE /identity/v3/projects/9732055c72d44d1da18d13a40a9fc8e5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:07.387820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:07.399052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa90bd-5d0f-404b-8ba0-1e4787da7fff service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:07.476126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:07.477017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:07.477301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:07.477595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] PATH_INFO: `/v3/users/fa672cde53354fc58003e97201fdd6b3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:07.478883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa672cde53354fc58003e97201fdd6b3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:07.482055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:07.534188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa672cde53354fc58003e97201fdd6b3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.535086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa672cde53354fc58003e97201fdd6b3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.544038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa672cde53354fc58003e97201fdd6b3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.544814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa672cde53354fc58003e97201fdd6b3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.554103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c85901e2-3210-4746-bd32-f47f0a4d0c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa672cde53354fc58003e97201fdd6b3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:07.560697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:07.584722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1080/2157] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:06:07 2022] DELETE /identity/v3/users/fa672cde53354fc58003e97201fdd6b3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:06:08.064335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.065873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:08.096883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a302-4721-4d65-a7c5-f77dce3cdba1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.165824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:08.177228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.178403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.178867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.179325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] PATH_INFO: `/v3/projects/e38a7363d9cf4257bca212e579d80542` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.180689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38a7363d9cf4257bca212e579d80542)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.182628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.255695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38a7363d9cf4257bca212e579d80542', 'request_id': 'req-39283fce-015c-414c-9493-f8feb057462c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:08.262393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38a7363d9cf4257bca212e579d80542', 'request_id': 'req-39283fce-015c-414c-9493-f8feb057462c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:08.273926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38a7363d9cf4257bca212e579d80542', 'request_id': 'req-39283fce-015c-414c-9493-f8feb057462c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:08.301351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] The token cache is being invalidate because project e38a7363d9cf4257bca212e579d80542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:06:08.301650 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:08.302311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-39283fce-015c-414c-9493-f8feb057462c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:08.303905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1078/2158] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:06:08 2022] DELETE /identity/v3/projects/e38a7363d9cf4257bca212e579d80542 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:08.310510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.310798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.312946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.369444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:08.370731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:08.443426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1081/2159] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:06:08.452305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.452596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.452932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.503799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:08.505368 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:08.514742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1079/2160] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:08.519147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.529756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389399f2-55d7-47c4-aa89-95c0e4bacb68 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.534880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.535246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.535562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.535846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.536583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:06:08.536920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.538530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.552009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1082/2161] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:06:08.560515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.573371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69d5b0-2919-4cb7-a120-2c06d0956e3b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.593463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.594285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.594600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.596239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.596239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.597997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.633932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1080/2162] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:06:08 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:06:08.634937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.650671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014b54a-605d-4048-b5cd-aad312b8a710 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.729486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.732823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.732823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.732823 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.733202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.735764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.760763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1083/2163] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:06:08 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:06:08.765070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.779283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cfd05-7e45-46cb-ae69-b61d68da16dc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.784675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.785572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.786101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.786390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.787156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.788853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.803300 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1081/2164] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:08.807353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.823908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701efd65-3b4e-4c1a-a7ca-3fa97438a6e6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.829540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7505d2-bbda-4fa2-9e93-7ae717ce06aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.830267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7505d2-bbda-4fa2-9e93-7ae717ce06aa admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.830560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7505d2-bbda-4fa2-9e93-7ae717ce06aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.830862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7505d2-bbda-4fa2-9e93-7ae717ce06aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.831670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7505d2-bbda-4fa2-9e93-7ae717ce06aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.833299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7505d2-bbda-4fa2-9e93-7ae717ce06aa admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.845530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1084/2165] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:08.849116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:08.860255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2731-31d0-43c2-9dbc-db00f7e1e18b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:08.865230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:08.866113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.866398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.866675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] PATH_INFO: `/v3/projects/22a42b2d9368442782907d739f29cc2e/users/e7cb35be298b495796bb586573ab3e27/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.887024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a42b2d9368442782907d739f29cc2e, user_id=e7cb35be298b495796bb586573ab3e27, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:08.889723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:08.905840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1082/2166] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:06:08 2022] PUT /identity/v3/projects/22a42b2d9368442782907d739f29cc2e/users/e7cb35be298b495796bb586573ab3e27/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:08.919546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fcc75-ecf2-4f35-9f02-e04cbd5bb509 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:08.919839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fcc75-ecf2-4f35-9f02-e04cbd5bb509 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:08.920121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fcc75-ecf2-4f35-9f02-e04cbd5bb509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:08.950222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-026fcc75-ecf2-4f35-9f02-e04cbd5bb509 None None] MFA Rules not processed for user `e7cb35be298b495796bb586573ab3e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:08.951851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-026fcc75-ecf2-4f35-9f02-e04cbd5bb509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:08.972931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:09.039928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1085/2167] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:09.052391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:09.054169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:09.119719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c04a25-3a22-4430-808a-f5a0a91fa9e5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:09.124937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:09.125683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:09.126042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:09.126397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:09.138394 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:09.140141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:09.150630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1083/2168] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:10.653306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:10.653811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:10.653936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:10.681668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:10.682809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:10.739461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1086/2169] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:10.752140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:10.753513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:10.785109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:10.825792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda91e6b-3ac4-4c6e-aea1-684f17dbb65f service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:10.830938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:10.831591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:10.831883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:10.832162 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:10.840890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:10.842555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:10.850802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1084/2170] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:06:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:06:12.796048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:12.798096 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:12.869319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294f686-6f88-4310-bec6-68c5f2c57e4d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:12.874808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:12.875713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:12.876024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:12.876322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] PATH_INFO: `/v3/users/05d51a9c4d8e4d869252835c37764abe` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:12.877515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d51a9c4d8e4d869252835c37764abe)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:12.879512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:12.924513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d51a9c4d8e4d869252835c37764abe'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:12.925162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d51a9c4d8e4d869252835c37764abe'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:12.932578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d51a9c4d8e4d869252835c37764abe'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:12.933321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d51a9c4d8e4d869252835c37764abe'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:12.935915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:12.943527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-51f8bb74-9371-4f3c-9ee9-449b59fc18cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d51a9c4d8e4d869252835c37764abe'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:12.961095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1087/2171] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:06:12 2022] DELETE /identity/v3/users/05d51a9c4d8e4d869252835c37764abe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:13.151306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.162745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d7fe-0cba-4fd7-bee2-4c925ed0d7b6 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.223326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.224184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.224492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.224771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] PATH_INFO: `/v3/projects/c37ecaa897a7485fb4a2ccdce8496eb9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.225935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37ecaa897a7485fb4a2ccdce8496eb9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.227908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.276353 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37ecaa897a7485fb4a2ccdce8496eb9', 'request_id': 'req-a1af0554-8efc-426e-87d9-7527097f7a22'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:13.284901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37ecaa897a7485fb4a2ccdce8496eb9', 'request_id': 'req-a1af0554-8efc-426e-87d9-7527097f7a22'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:13.291115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37ecaa897a7485fb4a2ccdce8496eb9', 'request_id': 'req-a1af0554-8efc-426e-87d9-7527097f7a22'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:13.304863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] The token cache is being invalidate because project c37ecaa897a7485fb4a2ccdce8496eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:06:13.305281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:13.305985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a1af0554-8efc-426e-87d9-7527097f7a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:13.307587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1085/2172] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:06:13 2022] DELETE /identity/v3/projects/c37ecaa897a7485fb4a2ccdce8496eb9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:13.312987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.313352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.313705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.331912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:13.333124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:13.407593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1088/2173] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:13.415024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.415321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.415607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.434720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:13.437248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:13.444340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1086/2174] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:13.448451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.458274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd885c50-27de-4d23-a724-9ea853183c7e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.462801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.463487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.463772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.464051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.465342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:06:13.465651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.467304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.479478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1089/2175] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:06:13.483735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.492732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b68cc5-4d6e-4b13-a51d-b23054796605 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.497356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.497993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.498277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.498557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.499273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.501199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.525366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1087/2176] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:06:13 2022] POST /identity/v3/projects => generated 371 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:06:13.530141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.540352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f553-51e4-410d-9f3e-b3fe743ea0a2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.596849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.597507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.597792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.598073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.598759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.600736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.620202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1090/2177] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:06:13 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:06:13.624587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.634870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ca793-35cf-45ef-a977-b7e1cb71680f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.640197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.640881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.641195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.641473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.642219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.643838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.654939 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1088/2178] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:13.658804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.668906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0375b-3799-46d0-aaf9-6a54ab659357 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.673477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761032e-a163-4812-8556-9b7e1517f58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.674127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761032e-a163-4812-8556-9b7e1517f58f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.674412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761032e-a163-4812-8556-9b7e1517f58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.674687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761032e-a163-4812-8556-9b7e1517f58f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.675440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761032e-a163-4812-8556-9b7e1517f58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.677032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761032e-a163-4812-8556-9b7e1517f58f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.688550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1091/2179] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:13.698310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.703950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618e471-6582-48a0-b925-150d2e93b1c1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.708738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.709632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.709927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.710204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] PATH_INFO: `/v3/projects/94dc7e8781f344b6b9329f256abe31ea/users/3a8ba70d455b4032b3027a5fceb4f2ec/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.714121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:13.745367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dc7e8781f344b6b9329f256abe31ea, user_id=3a8ba70d455b4032b3027a5fceb4f2ec, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.748618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.765967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1089/2180] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:06:13 2022] PUT /identity/v3/projects/94dc7e8781f344b6b9329f256abe31ea/users/3a8ba70d455b4032b3027a5fceb4f2ec/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:13.778687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4758b9-47a5-48f5-b3a4-4a40287936d6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.779034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4758b9-47a5-48f5-b3a4-4a40287936d6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.779983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4758b9-47a5-48f5-b3a4-4a40287936d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.812386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0e4758b9-47a5-48f5-b3a4-4a40287936d6 None None] MFA Rules not processed for user `3a8ba70d455b4032b3027a5fceb4f2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:13.813589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0e4758b9-47a5-48f5-b3a4-4a40287936d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:13.848087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:13.885353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1092/2181] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:13.897052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:13.898819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:13.955786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccddc1-f156-4ffd-a2ee-7a85a6c5fc84 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:13.959407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:13.960052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:13.960342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:13.960626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:13.970036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:13.971863 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:13.981081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1090/2182] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:23.882508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=14 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:23.976739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:27.656416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36de980-b660-48b5-abcb-157dcb4e2eb6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:27.656903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36de980-b660-48b5-abcb-157dcb4e2eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:27.656992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36de980-b660-48b5-abcb-157dcb4e2eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:27.675309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e36de980-b660-48b5-abcb-157dcb4e2eb6 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:27.676634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e36de980-b660-48b5-abcb-157dcb4e2eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:27.748651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1093/2183] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:06:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:27.761627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:27.764426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:27.845684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648cb4-c627-426e-b66c-e9efd1c13d48 service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:27.850728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:27.851450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:27.851736 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:27.852011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:27.861432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:27.863085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:27.875630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1091/2184] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:06:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:06:28.542461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087c6f5-9e72-42b2-ab01-f975243a93ec None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:28.542977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087c6f5-9e72-42b2-ab01-f975243a93ec None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:28.543686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087c6f5-9e72-42b2-ab01-f975243a93ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:28.573684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1087c6f5-9e72-42b2-ab01-f975243a93ec None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:28.576125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1087c6f5-9e72-42b2-ab01-f975243a93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:28.584691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1094/2185] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 12:06:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:28.598922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:28.611033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60477-6dab-463f-840b-391d57d74575 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:28.617064 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:28.617715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:28.618003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:28.618476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:28.628258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:28.629855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:28.639580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1092/2186] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:06:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:06:28.874888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e7d2b-9bbe-4052-80d0-4461fe7e05b7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:28.875404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e7d2b-9bbe-4052-80d0-4461fe7e05b7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:28.875611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e7d2b-9bbe-4052-80d0-4461fe7e05b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:28.895677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cb7e7d2b-9bbe-4052-80d0-4461fe7e05b7 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:28.897020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cb7e7d2b-9bbe-4052-80d0-4461fe7e05b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:28.906155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1095/2187] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:06:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:28.918096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:28.928576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822118d-ad9a-4637-a514-4e80ed4269cb service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:28.933670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:28.934354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:28.934636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:28.934913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:28.943385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:28.944973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:28.954275 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1093/2188] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:06:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:06:29.370092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:29.370381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:29.370660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:29.392505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:29.393780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:29.401304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1096/2189] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 12:06:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:29.414309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:29.425845 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3515-0939-4f6c-8902-3ecb59841c71 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:29.432553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e235ffb-043c-49da-9d1f-d9e3883a98b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:29.434606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e235ffb-043c-49da-9d1f-d9e3883a98b7 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:29.434933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e235ffb-043c-49da-9d1f-d9e3883a98b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:29.435316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e235ffb-043c-49da-9d1f-d9e3883a98b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:29.444629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e235ffb-043c-49da-9d1f-d9e3883a98b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:29.446313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e235ffb-043c-49da-9d1f-d9e3883a98b7 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:29.459686 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1094/2190] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:06:35.795433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:35.797192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:35.869063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba04fe7-4286-4a6e-8664-600132db8209 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:35.874152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:35.875050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:35.875337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:35.875616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] PATH_INFO: `/v3/users/601038b478b6408ab4a8e44ffc2d6cb7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:35.876819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=601038b478b6408ab4a8e44ffc2d6cb7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:35.878771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:35.925035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601038b478b6408ab4a8e44ffc2d6cb7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:35.925608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601038b478b6408ab4a8e44ffc2d6cb7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:35.932471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601038b478b6408ab4a8e44ffc2d6cb7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:35.933099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601038b478b6408ab4a8e44ffc2d6cb7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:35.941033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601038b478b6408ab4a8e44ffc2d6cb7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:35.955164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:35.962016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1097/2191] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:06:35 2022] DELETE /identity/v3/users/601038b478b6408ab4a8e44ffc2d6cb7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:35.966616 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:35.967251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:35.967391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:35.985191 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:35.986397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.046793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1095/2192] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:36.056322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.057728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.115225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d5762-99be-472e-b417-a0d14c61e77f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.118901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee858c8e-25c1-4bf2-81aa-70b9df268629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.119545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee858c8e-25c1-4bf2-81aa-70b9df268629 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.119828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee858c8e-25c1-4bf2-81aa-70b9df268629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.120111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee858c8e-25c1-4bf2-81aa-70b9df268629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.130045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee858c8e-25c1-4bf2-81aa-70b9df268629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.131514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee858c8e-25c1-4bf2-81aa-70b9df268629 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.141238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1098/2193] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:36.309756 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.312462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.325715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf0cf-d1e5-48c2-8e72-7619f53ef898 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.330821 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.331724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.332014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.332302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] PATH_INFO: `/v3/projects/a6725c8fde0e4bce93716183772e5dbd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.333510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6725c8fde0e4bce93716183772e5dbd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.335489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.383266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:36.386262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6725c8fde0e4bce93716183772e5dbd', 'request_id': 'req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:36.394047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6725c8fde0e4bce93716183772e5dbd', 'request_id': 'req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:36.401646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6725c8fde0e4bce93716183772e5dbd', 'request_id': 'req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:36.414228 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] The token cache is being invalidate because project a6725c8fde0e4bce93716183772e5dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:06:36.414516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:36.415143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fc44c2ac-fd50-4e32-9d5b-56e5983366fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:36.416666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1096/2194] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:06:36 2022] DELETE /identity/v3/projects/a6725c8fde0e4bce93716183772e5dbd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:36.422357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.422653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.422986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.440702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:36.441844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.505933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1099/2195] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:36.513716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.514245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.514596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.532905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:36.534087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.541779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1097/2196] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:36.546402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.556169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87ad0-25d5-4970-bc1c-58b3482e6ebc None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.560822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.561486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.561761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.562045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.562797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:06:36.563145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.564767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.576215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1100/2197] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:06:36.580416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.589505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8c01-1f5d-4ac4-966b-d91aabf53ee4 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.594999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.595648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.595930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.596206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.597154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.599056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.624287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1098/2198] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:06:36 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:06:36.628321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.638208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b19c4-c874-4d47-8f67-805074daaf90 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.694077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.694668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.694986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.695269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.695945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.697883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.717152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1101/2199] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:06:36 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:06:36.721140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.731520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51bf3d-1bb2-4bc6-a3c7-29bc5045bc5b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.736694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.737347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.737634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.737917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.738642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.740282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.750630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1099/2200] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:36.754361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.764448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44360cf7-ee48-48d4-aa03-e8c0f1239b5f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.769076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b5f81-4c9e-41b4-bda1-4343771ae5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.769736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5f81-4c9e-41b4-bda1-4343771ae5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.770022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5f81-4c9e-41b4-bda1-4343771ae5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.770297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5f81-4c9e-41b4-bda1-4343771ae5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.771050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b5f81-4c9e-41b4-bda1-4343771ae5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.772669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b5f81-4c9e-41b4-bda1-4343771ae5e1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.786684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1102/2201] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:36.787898 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.798734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842e6ed-cee0-4b56-b2f3-3023103b2cd9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.803489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.804363 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.804662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.804933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] PATH_INFO: `/v3/projects/43703500ab064d7091826ea7870273fb/users/567389c79da54aec809e004e4bde99e5/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.819781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43703500ab064d7091826ea7870273fb, user_id=567389c79da54aec809e004e4bde99e5, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:36.822460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:36.835070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1100/2202] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:06:36 2022] PUT /identity/v3/projects/43703500ab064d7091826ea7870273fb/users/567389c79da54aec809e004e4bde99e5/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:36.840262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.840735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.840944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:36.855455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:36.861922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:36.863155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.922603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1103/2203] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:36.932140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:36.933833 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:36.991025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b086-cce1-4038-9c7f-214d1e2ac4e8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:36.994665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd157c1-7759-4cd8-9184-4d88acf69fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:36.995319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd157c1-7759-4cd8-9184-4d88acf69fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:36.995608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd157c1-7759-4cd8-9184-4d88acf69fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:36.995922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd157c1-7759-4cd8-9184-4d88acf69fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:37.005696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd157c1-7759-4cd8-9184-4d88acf69fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:37.007190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd157c1-7759-4cd8-9184-4d88acf69fc0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:37.018040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1101/2204] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:40.553850 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:40.555381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:40.573899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f18fc3-10dd-4f8f-962f-d90783531530 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:40.575606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:40.576582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:40.576871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:40.577151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] PATH_INFO: `/v3/users/e7cb35be298b495796bb586573ab3e27` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:40.578450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7cb35be298b495796bb586573ab3e27)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:40.580457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:40.635305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7cb35be298b495796bb586573ab3e27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:40.636075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7cb35be298b495796bb586573ab3e27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:40.645911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7cb35be298b495796bb586573ab3e27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:40.646830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7cb35be298b495796bb586573ab3e27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:40.657495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7cb35be298b495796bb586573ab3e27'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:40.676828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1104/2205] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:06:40 2022] DELETE /identity/v3/users/e7cb35be298b495796bb586573ab3e27 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:40.681540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff757-2f9d-4179-b363-67ff8df87007 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:40.682001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff757-2f9d-4179-b363-67ff8df87007 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:40.682105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff757-2f9d-4179-b363-67ff8df87007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:40.706834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-306ff757-2f9d-4179-b363-67ff8df87007 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:40.708403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-306ff757-2f9d-4179-b363-67ff8df87007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:40.787886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1102/2206] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:40.820712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:40.827138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:40.917955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddb990-0f4a-4c3f-a734-f4b9f329a61e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:40.921989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:40.922990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:40.923149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:40.923437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:40.934030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:40.936328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:40.947368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1105/2207] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:41.344908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.345371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.345698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.361958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.364069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:41.378748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37ea91-947f-4509-a63d-bb5b86877848 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.384244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:41.388473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.388618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] MFA Rules not processed for user `567389c79da54aec809e004e4bde99e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:41.389473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.389767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.390053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] PATH_INFO: `/v3/projects/22a42b2d9368442782907d739f29cc2e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.390752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:41.391913 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a42b2d9368442782907d739f29cc2e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.394032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.427044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:41.465392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a42b2d9368442782907d739f29cc2e', 'request_id': 'req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:41.475990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a42b2d9368442782907d739f29cc2e', 'request_id': 'req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:41.482889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1103/2208] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:41.487620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.489455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:41.491911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a42b2d9368442782907d739f29cc2e', 'request_id': 'req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:41.504870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] The token cache is being invalidate because project 22a42b2d9368442782907d739f29cc2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:06:41.505266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:41.506260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:06:41.507247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58844239-3118-49d3-80d9-b12d69db0fbc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.509959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1106/2209] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:06:41 2022] DELETE /identity/v3/projects/22a42b2d9368442782907d739f29cc2e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:41.578011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.578718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.579042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.579328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.580052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.582097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.615139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1104/2210] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:06:41 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:06:41.619724 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.621457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:41.637029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccaa9c-2dfc-4752-aac8-c8b5453d4dea admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.713194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.714100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.714318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.714597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.715333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.717866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.739818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1107/2211] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:06:41 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:06:41.743855 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.754125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d872f1-1c6a-4130-8d9a-311a8b7dd0de admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.758856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.759520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.759809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.760096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.760870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.762910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.773760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1105/2212] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:41.777929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.788445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7653b7c-25b2-4a10-8d4b-c1b3beb360fe admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.793170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.794044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.794336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.795288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] PATH_INFO: `/v3/projects/c2bad9bd51b247e78dbf42c6c6ab4528/users/7ff3b7de582749369390300f57fd7503/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.810661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bad9bd51b247e78dbf42c6c6ab4528, user_id=7ff3b7de582749369390300f57fd7503, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.813422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.827326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1108/2213] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:06:41 2022] PUT /identity/v3/projects/c2bad9bd51b247e78dbf42c6c6ab4528/users/7ff3b7de582749369390300f57fd7503/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:41.831379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.842413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844a107-ed4a-4315-991e-c3cf5827a7ca admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.900614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e606c-9e11-4e50-b454-ab615a3706b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.901282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e606c-9e11-4e50-b454-ab615a3706b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.901574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e606c-9e11-4e50-b454-ab615a3706b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.901857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e606c-9e11-4e50-b454-ab615a3706b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.902604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e606c-9e11-4e50-b454-ab615a3706b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.904841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e606c-9e11-4e50-b454-ab615a3706b6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.914460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1106/2214] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:06:41.918641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:41.929743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92327307-60d6-465f-a1d9-e1b5bed92b2a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:41.934369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:41.935439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:41.935702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:41.935982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] PATH_INFO: `/v3/projects/c2bad9bd51b247e78dbf42c6c6ab4528/users/7ff3b7de582749369390300f57fd7503/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:41.941583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bad9bd51b247e78dbf42c6c6ab4528, user_id=7ff3b7de582749369390300f57fd7503, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:41.945188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:41.956345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1109/2215] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:06:41 2022] PUT /identity/v3/projects/c2bad9bd51b247e78dbf42c6c6ab4528/users/7ff3b7de582749369390300f57fd7503/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:06:47.200507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319425c1-b6cc-4d5e-986e-8e83d80bf7a4 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:47.201446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319425c1-b6cc-4d5e-986e-8e83d80bf7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:47.202886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319425c1-b6cc-4d5e-986e-8e83d80bf7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:47.237018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:47.240470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-319425c1-b6cc-4d5e-986e-8e83d80bf7a4 None None] MFA Rules not processed for user `7ff3b7de582749369390300f57fd7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:47.242385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-319425c1-b6cc-4d5e-986e-8e83d80bf7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:47.317588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1107/2216] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:47.332921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:06:47.334812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:47.396311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:47.431618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085dadb-784a-4690-a378-a2a90d33660b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:06:47.437643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:06:47.438279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:47.438563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:47.438871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:47.440613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:47.496133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:06:47.497730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:06:47.507173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1110/2217] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:06:48.569295 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c01cc6-4ff4-4deb-8311-7cc4c712b950 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:06:48.569843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c01cc6-4ff4-4deb-8311-7cc4c712b950 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:06:48.569843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c01cc6-4ff4-4deb-8311-7cc4c712b950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:06:48.591771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-07c01cc6-4ff4-4deb-8311-7cc4c712b950 None None] MFA Rules not processed for user `567389c79da54aec809e004e4bde99e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:06:48.593263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-07c01cc6-4ff4-4deb-8311-7cc4c712b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:06:48.601107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1108/2218] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:06:57.503107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=39 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:06:58.583907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=31 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:02.943300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:02.944757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:03.011427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200812a-7659-4dce-9487-05dc1cf3df37 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.016680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.017552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.017835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.018108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] PATH_INFO: `/v3/users/3a8ba70d455b4032b3027a5fceb4f2ec` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.019328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8ba70d455b4032b3027a5fceb4f2ec)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.021266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.067770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8ba70d455b4032b3027a5fceb4f2ec'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.068416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8ba70d455b4032b3027a5fceb4f2ec'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.075181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8ba70d455b4032b3027a5fceb4f2ec'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.075811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8ba70d455b4032b3027a5fceb4f2ec'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.083505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8ba70d455b4032b3027a5fceb4f2ec'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.102488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1111/2219] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:07:02 2022] DELETE /identity/v3/users/3a8ba70d455b4032b3027a5fceb4f2ec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:07:03.106983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.107436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.107545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.124977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:03.126187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:03.184782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1109/2220] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:07:03.194750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:03.196193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:03.252954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad113a-2da6-4d84-8610-249182ab9a09 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.256598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dfaab-f198-41a2-b57d-677220c857a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.257246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dfaab-f198-41a2-b57d-677220c857a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.257535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dfaab-f198-41a2-b57d-677220c857a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.257820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dfaab-f198-41a2-b57d-677220c857a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.267596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dfaab-f198-41a2-b57d-677220c857a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.269095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dfaab-f198-41a2-b57d-677220c857a1 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.278477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1112/2221] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:07:03.550705 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:03.552209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:03.564793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82f04a-5e99-4acf-991d-22a94bd2ea6e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.569902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.570802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.571149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.571429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] PATH_INFO: `/v3/projects/94dc7e8781f344b6b9329f256abe31ea` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.572695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=94dc7e8781f344b6b9329f256abe31ea)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.574741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.623342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94dc7e8781f344b6b9329f256abe31ea', 'request_id': 'req-0008b117-9547-4485-bfcc-0e145053c35b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.629618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94dc7e8781f344b6b9329f256abe31ea', 'request_id': 'req-0008b117-9547-4485-bfcc-0e145053c35b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.635642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94dc7e8781f344b6b9329f256abe31ea', 'request_id': 'req-0008b117-9547-4485-bfcc-0e145053c35b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.648015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] The token cache is being invalidate because project 94dc7e8781f344b6b9329f256abe31ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:07:03.648289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.648902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0008b117-9547-4485-bfcc-0e145053c35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:07:03.650426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1110/2222] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:07:03 2022] DELETE /identity/v3/projects/94dc7e8781f344b6b9329f256abe31ea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:07:03.656737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.657032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.657317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.674458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:03.675704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:03.693346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:03.738964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1113/2223] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:07:03.746523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.746819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.747138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.766553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:03.767800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:03.775661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1111/2224] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:07:03.779734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:03.789439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b663-ae30-4357-8bc4-f7f8fdc3926a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.793946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.794599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.794923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.795212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.795936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:07:03.796862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.798403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.809831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1114/2225] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:07:03.814903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:03.824003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149cc5-154e-47b8-95e8-63a9b9896ee6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.829666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.830311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.830596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.830900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.831594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.833513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.859656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1112/2226] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:07:03 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:07:03.863767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:03.873575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97464099-7feb-41a8-b88c-7e8edba7ed3c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.929501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.930165 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.930453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.930737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.931467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.933420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.953220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1115/2227] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:07:03 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:07:03.957144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:03.967401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e001b8-0cfe-4b78-bcfb-e9c975fe889c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:03.973014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:03.973662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:03.973950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:03.974230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:03.974984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:03.976576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:03.988039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1113/2228] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:07:03.991772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:04.002485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce025950-247e-4a30-aa93-295d5c0575c0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:04.007113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667810-f497-40a6-b69c-70b125f2c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:04.007787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667810-f497-40a6-b69c-70b125f2c913 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:04.008071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667810-f497-40a6-b69c-70b125f2c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:04.008462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667810-f497-40a6-b69c-70b125f2c913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:04.009115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd667810-f497-40a6-b69c-70b125f2c913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:04.010861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd667810-f497-40a6-b69c-70b125f2c913 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:04.023516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1116/2229] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:07:04.027636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:04.037831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc615c-04ee-4c62-9658-1d35a6553d8b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:04.042354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:04.043258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:04.043544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:04.043822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] PATH_INFO: `/v3/projects/c80560ca253e41239020e2c008a9b3ba/users/adb92379209e46e79de4df370183fa8a/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:04.058533 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80560ca253e41239020e2c008a9b3ba, user_id=adb92379209e46e79de4df370183fa8a, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:04.061223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:04.077238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1114/2230] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:07:04 2022] PUT /identity/v3/projects/c80560ca253e41239020e2c008a9b3ba/users/adb92379209e46e79de4df370183fa8a/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:07:04.082330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579bdba-6efe-41b0-9334-d19f6a4e93a5 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:04.082910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579bdba-6efe-41b0-9334-d19f6a4e93a5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:04.083301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579bdba-6efe-41b0-9334-d19f6a4e93a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:04.103099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6579bdba-6efe-41b0-9334-d19f6a4e93a5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:04.104296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6579bdba-6efe-41b0-9334-d19f6a4e93a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:04.165561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1117/2231] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:07:04.175494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:04.177359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:04.199894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:04.237895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baf50c-b49a-4ab8-a559-2224739cf25d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:04.241588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:04.242236 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:04.242519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:04.242798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:04.252524 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:04.253948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:04.263403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1115/2232] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:07:08.308330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.308874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.308874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.342216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] MFA Rules not processed for user `adb92379209e46e79de4df370183fa8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:08.344292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:08.365983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:08.427518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1118/2233] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:07:08.431890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:08.443716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe06fec-b974-46b4-a1ec-443e77958601 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:08.448877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:08.449757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.450222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.450509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.451431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:08.453429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:08.484812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1116/2234] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:07:08 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:07:08.489785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:08.504356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753706-daca-4ac8-9936-57311b811e56 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:08.568677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:08.569373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.569658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.569957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.570668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:08.572710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:08.600270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1119/2235] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:07:08 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:07:08.604137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:08.616117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade2430-8ad2-4f5f-b9de-0f87889b0829 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:08.621223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:08.621896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.622186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.622465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.623372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:08.625062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:08.639698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1117/2236] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:07:08.640977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:08.651465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14258e59-d786-4a82-ad0e-02271e669682 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:08.659495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:08.663250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.663668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.664053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] PATH_INFO: `/v3/projects/c2be2e7a09154df2a1c2b1bbc17c1094/users/7d6cea527f36408989e27a6ca577371e/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.681372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2be2e7a09154df2a1c2b1bbc17c1094, user_id=7d6cea527f36408989e27a6ca577371e, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:08.684149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:08.699627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1120/2237] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:07:08 2022] PUT /identity/v3/projects/c2be2e7a09154df2a1c2b1bbc17c1094/users/7d6cea527f36408989e27a6ca577371e/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:07:08.703631 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:08.713744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e2f0-0d90-4ecb-ba83-5b9f3028ce46 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:08.777414 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57f8de-2d3d-4f37-bdcf-0a3583a7a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:08.778083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f8de-2d3d-4f37-bdcf-0a3583a7a1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.778369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f8de-2d3d-4f37-bdcf-0a3583a7a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.778652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f8de-2d3d-4f37-bdcf-0a3583a7a1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.779400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57f8de-2d3d-4f37-bdcf-0a3583a7a1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:08.782053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57f8de-2d3d-4f37-bdcf-0a3583a7a1e8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:08.791945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1118/2238] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:07:08.796706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:08.807384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce817bc9-92a3-4616-8788-c99b394d57c3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:08.813567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:08.814496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:08.814784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:08.815109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] PATH_INFO: `/v3/projects/c2be2e7a09154df2a1c2b1bbc17c1094/users/7d6cea527f36408989e27a6ca577371e/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:08.817119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2be2e7a09154df2a1c2b1bbc17c1094, user_id=7d6cea527f36408989e27a6ca577371e, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:08.819956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:08.831521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1121/2239] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:07:08 2022] PUT /identity/v3/projects/c2be2e7a09154df2a1c2b1bbc17c1094/users/7d6cea527f36408989e27a6ca577371e/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:07:13.452483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:13.453065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:13.453065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:13.493145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] MFA Rules not processed for user `7d6cea527f36408989e27a6ca577371e`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:13.494436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:13.582060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1119/2240] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:07:13.600959 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:13.604127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:13.688668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38519d3-24c6-4326-aa91-7cbd32933b75 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:13.694484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af361eee-4f8f-49aa-a1c0-da469a5dde85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:13.695308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af361eee-4f8f-49aa-a1c0-da469a5dde85 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:13.695667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af361eee-4f8f-49aa-a1c0-da469a5dde85 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:13.696028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af361eee-4f8f-49aa-a1c0-da469a5dde85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:13.702586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:13.760594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af361eee-4f8f-49aa-a1c0-da469a5dde85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:13.762230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af361eee-4f8f-49aa-a1c0-da469a5dde85 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:13.773459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1122/2241] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:07:17.929210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:17.930609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:17.943937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:18.025107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc29b-abc3-417a-a2ef-15b379f927db None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:18.031024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:18.031679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:18.031967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:18.032248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:18.033986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:18.046043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:18.050123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:18.062667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1120/2242] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:07:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:07:23.768893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=24 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:28.057438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=36 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:32.560870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f4aa-dd20-4d0e-b574-b42286af53af None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:32.561352 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f4aa-dd20-4d0e-b574-b42286af53af None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:32.561449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f4aa-dd20-4d0e-b574-b42286af53af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:32.592509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0c87f4aa-dd20-4d0e-b574-b42286af53af None None] MFA Rules not processed for user `6aebb057a324492699141a878793c097`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:32.592835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0c87f4aa-dd20-4d0e-b574-b42286af53af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:32.665741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1123/2243] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:07:36.820393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:07:36.881636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfbdbf-1008-43a1-b724-4e73f7163f37 service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:07:36.885357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:07:36.886003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:36.886292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:36.886572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:36.888535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:36.959397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:07:36.960876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21309120-9fd0-454b-9438-c9ef1be412b3 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:07:36.975781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1121/2244] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:07:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:07:42.658386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=25 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:42.768198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:07:42.768491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:07:42.768776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:07:42.788936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] MFA Rules not processed for user `adb92379209e46e79de4df370183fa8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:07:42.790342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:07:42.799239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1124/2245] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:07:46.966504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=50 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:07:52.782265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:08:00.200723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:00.201250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:00.201340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:00.255165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:08:00.256392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:00.333882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1122/2246] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:08:00.392621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:00.394464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:00.491232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736fc1b-d32d-41e8-be57-46d814d163f1 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:00.496444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c15054-d33c-4fdd-a8fc-d26cf4964796 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:00.497091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c15054-d33c-4fdd-a8fc-d26cf4964796 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:00.497381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c15054-d33c-4fdd-a8fc-d26cf4964796 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:00.497665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c15054-d33c-4fdd-a8fc-d26cf4964796 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:00.505728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c15054-d33c-4fdd-a8fc-d26cf4964796 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:00.507364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c15054-d33c-4fdd-a8fc-d26cf4964796 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:00.515958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1125/2247] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 12:08:05.801540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:05.826913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1d8a4-5f2c-42a1-a870-ef25a4379d9a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:05.835233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998c5f-75c2-4f7a-a1ad-2bbb1cd7e113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:05.836065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998c5f-75c2-4f7a-a1ad-2bbb1cd7e113 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:05.836429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998c5f-75c2-4f7a-a1ad-2bbb1cd7e113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:05.836757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998c5f-75c2-4f7a-a1ad-2bbb1cd7e113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:05.854020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998c5f-75c2-4f7a-a1ad-2bbb1cd7e113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:05.854789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998c5f-75c2-4f7a-a1ad-2bbb1cd7e113 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:05.876687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1123/2248] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:10.512139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=41 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:08:15.189073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20010d-eede-44a7-ae09-c4f7c00300f5 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.189611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20010d-eede-44a7-ae09-c4f7c00300f5 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.189703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20010d-eede-44a7-ae09-c4f7c00300f5 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.191254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1126/2249] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:08:15.198396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829095ae-1638-497e-8a93-2afa540c2be5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.198928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829095ae-1638-497e-8a93-2afa540c2be5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.199203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829095ae-1638-497e-8a93-2afa540c2be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.237049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-829095ae-1638-497e-8a93-2afa540c2be5 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:08:15.238166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-829095ae-1638-497e-8a93-2afa540c2be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:15.247860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1124/2250] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:08:15.260109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20010d-eede-44a7-ae09-c4f7c00300f5 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:15.261861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4c20010d-eede-44a7-ae09-c4f7c00300f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:15.342624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20010d-eede-44a7-ae09-c4f7c00300f5 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:15.346228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f84c3e-bc0d-4a25-a29d-84bcc85b7361 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:15.346904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84c3e-bc0d-4a25-a29d-84bcc85b7361 service glance] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.347193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84c3e-bc0d-4a25-a29d-84bcc85b7361 service glance] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.347476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84c3e-bc0d-4a25-a29d-84bcc85b7361 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.358361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f84c3e-bc0d-4a25-a29d-84bcc85b7361 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:15.359879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f84c3e-bc0d-4a25-a29d-84bcc85b7361 service glance] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:15.372216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1127/2251] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:15.440558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e0dfd-a2ba-465b-a396-f479a85e036d None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.446150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e0dfd-a2ba-465b-a396-f479a85e036d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.446592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e0dfd-a2ba-465b-a396-f479a85e036d None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.448181 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1125/2252] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:08:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:08:15.454023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1656d-ec3a-4268-8dbf-3b98bd65a2bf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.454461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1656d-ec3a-4268-8dbf-3b98bd65a2bf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.454737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1656d-ec3a-4268-8dbf-3b98bd65a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.498262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-77c1656d-ec3a-4268-8dbf-3b98bd65a2bf None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:08:15.499429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-77c1656d-ec3a-4268-8dbf-3b98bd65a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:15.505948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1128/2253] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:08:15.517953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e0dfd-a2ba-465b-a396-f479a85e036d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:15.530237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e0dfd-a2ba-465b-a396-f479a85e036d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:15.533913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8fb04-8a78-45b9-ab56-41ef541eb3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:15.534542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8fb04-8a78-45b9-ab56-41ef541eb3de service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.534826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8fb04-8a78-45b9-ab56-41ef541eb3de service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.535269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8fb04-8a78-45b9-ab56-41ef541eb3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.543237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8fb04-8a78-45b9-ab56-41ef541eb3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:15.544651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8fb04-8a78-45b9-ab56-41ef541eb3de service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:15.552334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1126/2254] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:15.633780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d524f4-90c8-4664-9462-1e88fc800dc7 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.634076 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d524f4-90c8-4664-9462-1e88fc800dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.634397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d524f4-90c8-4664-9462-1e88fc800dc7 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.635809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1129/2255] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:08:15.641035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4692c-ec86-4d09-95d7-afa52cd1473e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.641331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4692c-ec86-4d09-95d7-afa52cd1473e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.641614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4692c-ec86-4d09-95d7-afa52cd1473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.660146 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7de4692c-ec86-4d09-95d7-afa52cd1473e None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:08:15.661349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7de4692c-ec86-4d09-95d7-afa52cd1473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:15.671147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1127/2256] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:08:15.681464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d524f4-90c8-4664-9462-1e88fc800dc7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:15.692487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d524f4-90c8-4664-9462-1e88fc800dc7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:15.696079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:15.697407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:15.697694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:15.697975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:15.708677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:15.710125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:15.722120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1130/2257] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:18.525467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:18.525935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:18.526132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:18.544256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:08:18.545446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:18.552757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1128/2258] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:08:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:08:18.564431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:18.575450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09f1ad-715e-4729-859b-a94e5997c224 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:18.580325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:18.580949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:18.581235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:18.581515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:18.590177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:18.592077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:18.600048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1131/2259] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:08:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:08:28.538384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=67 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:08:28.596934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=56 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:08:43.557956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:43.567534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa0c08-1d56-413d-8c3f-7862935adc56 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:43.571335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:43.572001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:43.572425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:43.572711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:43.574497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:43.589323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:43.591817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f908fc2-0f83-4de8-8e1d-e902a3629b67 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:43.608281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1129/2260] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:48.343854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:48.353871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7c6bf-0fa2-48f9-aa82-ee0effb5aa02 service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:48.357433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:48.358058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:48.358341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:48.358619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:48.360402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:48.374193 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:48.375676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:48.386496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1132/2261] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:49.277906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:49.278364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:49.278473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:49.296057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:08:49.297256 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:49.304390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1130/2262] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 12:08:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:08:49.316675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:49.328697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bc365-69d8-4380-b91e-cf53ef05a992 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:49.333592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:49.334222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:49.334506 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:49.334784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:49.343312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:49.344887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:49.353161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1133/2263] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:08:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:08:55.980240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:08:55.992646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b806954-f687-4e6b-bdc5-a682dd355a75 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:08:55.998909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:08:55.999544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:08:55.999827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:08:56.000105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:08:56.001872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:08:56.062071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:08:56.063708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aefbab-07e4-4309-9572-66d90da322de service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:08:56.073179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1131/2264] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:08:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:08:59.349578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=12 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:09:02.945000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:02.946555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:02.958681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd8e1-2c66-4260-97e7-49ae43a17c44 service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:02.962234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:02.962910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:02.963199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:02.963566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:02.965606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:02.975342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:02.976797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:02.985966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1134/2265] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:05.586182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:05.586688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:05.586801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:05.604708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] MFA Rules not processed for user `473532c21cad4c3492c8425855993b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:09:05.605884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:05.612636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1132/2266] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:09:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:09:11.828883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:11.830404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:11.848822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941520cc-9574-4b71-85cb-04af3557506e service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:11.854442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:11.855120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:11.855408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:11.855687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:11.857472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:11.869590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:11.871384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:11.882046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1135/2267] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:15.599355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=40 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:09:21.877719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=12 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:09:29.456512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:29.469580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08012b5c-6cb0-405c-a9bb-1f0a5bbd13b6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:29.474993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:29.475626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:29.475913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:29.476195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:29.477948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:29.538142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:29.539762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:29.550209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1133/2268] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:34.073030 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:34.084184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc56264-4334-450a-af68-93138a54de12 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:34.090660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:34.091337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:34.091621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:34.091899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:34.093655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:34.104210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:34.105796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da26b9-ba46-4ca4-a415-d7b4f3fe6ff2 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:34.114896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1136/2269] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:37.502344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:37.514340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586935a7-8ead-4a44-9077-5001040b75bf service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:37.520455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:37.530446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:37.530647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:37.532425 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:37.535271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:37.550990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:37.552572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:37.566357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1134/2270] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:44.110558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:09:46.251326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:46.251920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:46.251920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:46.269641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] MFA Rules not processed for user `6aebb057a324492699141a878793c097`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:09:46.270872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:46.277837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1137/2271] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:09:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:09:46.612078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:46.621829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb755f-f11c-4164-a6ff-ea95fc714648 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:46.625735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:46.626485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:46.626870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:46.627155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:46.628947 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:46.640523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:46.642485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:46.651681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1135/2272] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:48.022699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:48.025005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:48.041450 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a49112-e921-4487-a22b-5eabdab8cee2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:48.046729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:48.047436 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:48.047726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:48.048027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:48.049861 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:48.063128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:48.064775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:48.078243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1138/2273] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:09:56.647259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=40 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:09:57.747096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:09:57.760769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc5f21-f904-48a5-aae6-07f6c2e292c8 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:09:57.766202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:09:57.766857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:09:57.767149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:09:57.767432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:09:57.769158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:09:57.828257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:09:57.829857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64509c-3264-45c4-b6b8-7bbc89420a63 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:09:57.840501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1136/2274] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:09:58.072226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=9 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:09:59.990472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:00.007232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961878c-cd60-4124-bf0b-9240efd2b63b service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:00.013790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:00.014448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:00.014735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:00.015044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:00.016848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:00.030746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:00.032410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:00.043003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1139/2275] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:09:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:07.098971 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:07.099497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:07.099613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:07.118555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] MFA Rules not processed for user `c8bda0c079054e198ffeca4c0ac18bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:07.119836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:07.128594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1137/2276] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:07.432658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:07.442020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826badf-8189-4814-8090-8f8a822d958b service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:07.446679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:07.447347 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:07.447633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:07.448018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:07.450298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:07.459706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:07.462121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd309711-4c36-4855-b5ee-b4505eb28e5b service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:07.470073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1140/2277] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:17.112816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=31 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:17.466620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=12 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:34.420651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:34.432048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017d951-5b89-41a7-80ac-74a4a4883356 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:34.435813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:34.436542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.436929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.437276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.439399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:34.514712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:34.516200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bc6e9-a7b8-4a33-b2c2-d9491ad14e9b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:34.531062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1138/2278] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:34.561617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41734e-6b22-4f99-afb8-21f595985526 None None] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.562196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41734e-6b22-4f99-afb8-21f595985526 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.562196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41734e-6b22-4f99-afb8-21f595985526 None None] PATH_INFO: `/` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.563625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1141/2279] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:10:34.570897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b353c9-563f-4d4f-b2db-8413da231165 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.571572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b353c9-563f-4d4f-b2db-8413da231165 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.571850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b353c9-563f-4d4f-b2db-8413da231165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.590481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a2b353c9-563f-4d4f-b2db-8413da231165 None None] MFA Rules not processed for user `2665b33ffa38466a98a5e4ea838916c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:34.591675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a2b353c9-563f-4d4f-b2db-8413da231165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:34.598481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1139/2280] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:10:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:34.612258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41734e-6b22-4f99-afb8-21f595985526 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:34.625237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41734e-6b22-4f99-afb8-21f595985526 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:34.632418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc9234-2328-4c82-9fad-ec529dd2e8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:34.633126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc9234-2328-4c82-9fad-ec529dd2e8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.633420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc9234-2328-4c82-9fad-ec529dd2e8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.633731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc9234-2328-4c82-9fad-ec529dd2e8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.651110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc9234-2328-4c82-9fad-ec529dd2e8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:34.652904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc9234-2328-4c82-9fad-ec529dd2e8fe service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:34.661514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1142/2281] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:34.748502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eaabc-9c49-4235-955a-05b3a934ef5a None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.748797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eaabc-9c49-4235-955a-05b3a934ef5a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.749298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eaabc-9c49-4235-955a-05b3a934ef5a None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.750709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1140/2282] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:10:34.756842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.757160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.757448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.777263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:34.778804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:34.789635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1143/2283] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:10:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:34.802012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eaabc-9c49-4235-955a-05b3a934ef5a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:34.814128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eaabc-9c49-4235-955a-05b3a934ef5a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:34.817910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:34.818556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:34.818871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:34.819215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:34.830254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:34.831794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:34.843092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1141/2284] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:38.137622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:38.151635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f96b55-32ca-422d-869a-549817692cc2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:38.162228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:38.164012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.164612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.165107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.167497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:38.188130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:38.189973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:38.200609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:38.203258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:38.220447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1144/2285] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:38.230197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f23e0-b35c-4b4a-bd51-48da1f680905 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:38.235516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:38.236396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.236695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.236960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] PATH_INFO: `/v3/users/473532c21cad4c3492c8425855993b0f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.238188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=473532c21cad4c3492c8425855993b0f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:38.240374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:38.301891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473532c21cad4c3492c8425855993b0f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.302550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473532c21cad4c3492c8425855993b0f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.303321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473532c21cad4c3492c8425855993b0f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.311605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473532c21cad4c3492c8425855993b0f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.324704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-1a8e91db-bda4-4972-bf2b-5a5ef006d3cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473532c21cad4c3492c8425855993b0f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.337885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1142/2286] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:10:38 2022] DELETE /identity/v3/users/473532c21cad4c3492c8425855993b0f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:38.524396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:38.535163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a27691-d319-445d-b7b5-c362ac20e1ba service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:38.600219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:38.601073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.601357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.601636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] PATH_INFO: `/v3/projects/cf5de9215c3243e1b2fabc859f102b2a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.602802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5de9215c3243e1b2fabc859f102b2a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:38.604737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:38.652554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5de9215c3243e1b2fabc859f102b2a', 'request_id': 'req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.658662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5de9215c3243e1b2fabc859f102b2a', 'request_id': 'req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.675237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5de9215c3243e1b2fabc859f102b2a', 'request_id': 'req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.690708 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] The token cache is being invalidate because project cf5de9215c3243e1b2fabc859f102b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:38.691082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.691693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-89f350b8-b8eb-4a36-8286-7e3ad1647e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:38.693268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1145/2287] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:38 2022] DELETE /identity/v3/projects/cf5de9215c3243e1b2fabc859f102b2a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:10:38.699012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.699306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.699589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.717909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:38.719147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:38.763081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:38.783763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1143/2288] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:38.791090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.791377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.791664 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.810261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:38.811485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:38.820355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1146/2289] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:38.824530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:38.834033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1ee45-c5df-4722-86ac-3952bdb7b140 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:38.839521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:38.840165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.840450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.840730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.841447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:10:38.841761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:38.843535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:38.855323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1144/2290] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:10:38.859535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:38.869402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2658f1-8f7c-46d2-9c62-de4c681b6b00 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:38.874212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:38.874881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.875176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.875459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.876178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:38.878148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:38.913066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1147/2291] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:38 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:38.916962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:38.926791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314c085-9d8a-4454-a8fa-967d62aee58d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:38.984044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:38.984711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:38.984999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:38.985281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:38.985968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:38.987929 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:39.009923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1145/2292] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:10:38 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:39.014001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:39.025137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33abcf-97d3-40ae-b66d-be37ead42952 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:39.029807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:39.030474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:39.030790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:39.031129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:39.031949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:39.033574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:39.043711 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1148/2293] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:39.047651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:39.058390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6992850-4039-4419-ad99-aeb1cf69e166 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:39.062884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9b686-9033-496a-9d9c-b13883b9fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:39.063537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b686-9033-496a-9d9c-b13883b9fae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:39.063825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b686-9033-496a-9d9c-b13883b9fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:39.064107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b686-9033-496a-9d9c-b13883b9fae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:39.064822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9b686-9033-496a-9d9c-b13883b9fae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:39.066422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9b686-9033-496a-9d9c-b13883b9fae9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:39.077324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1146/2294] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:39.081315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:39.091900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c94e6-4106-4130-a620-f66ca8838ba1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:39.096395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:39.097289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:39.097567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:39.097839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] PATH_INFO: `/v3/projects/d7673fc376a940ee95c9381d83a5393f/users/71686b16188c45b0baeb13493581ebd9/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:39.101294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:39.114223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7673fc376a940ee95c9381d83a5393f, user_id=71686b16188c45b0baeb13493581ebd9, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:39.117665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:39.130951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1149/2295] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:39 2022] PUT /identity/v3/projects/d7673fc376a940ee95c9381d83a5393f/users/71686b16188c45b0baeb13493581ebd9/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:39.142774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:39.143097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:39.143380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:39.169881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] MFA Rules not processed for user `71686b16188c45b0baeb13493581ebd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:39.171043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:39.238080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1147/2296] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:39.249285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:39.251060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:39.322775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34333154-ab3a-41b7-84e1-58d9a0e0fd95 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:39.328299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce54ff4-1cdc-4f2d-af72-1bdc7877e002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:39.328940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54ff4-1cdc-4f2d-af72-1bdc7877e002 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:39.329225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54ff4-1cdc-4f2d-af72-1bdc7877e002 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:39.329505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54ff4-1cdc-4f2d-af72-1bdc7877e002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:39.337393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce54ff4-1cdc-4f2d-af72-1bdc7877e002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:39.339014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce54ff4-1cdc-4f2d-af72-1bdc7877e002 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:39.347878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1150/2297] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:42.011308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:42.056352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:42.075895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ba64-774a-4298-aad6-60be79c6c6a1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:42.080315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:42.081174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.081463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.081741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] PATH_INFO: `/v3/users/71686b16188c45b0baeb13493581ebd9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.082928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=71686b16188c45b0baeb13493581ebd9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:42.085218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:42.134825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71686b16188c45b0baeb13493581ebd9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.135526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71686b16188c45b0baeb13493581ebd9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.136143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71686b16188c45b0baeb13493581ebd9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.143497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71686b16188c45b0baeb13493581ebd9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.156341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71686b16188c45b0baeb13493581ebd9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.170764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1148/2298] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:10:42 2022] DELETE /identity/v3/users/71686b16188c45b0baeb13493581ebd9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:42.175097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.175553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.175836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.193690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:42.195699 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:42.260514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1151/2299] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:42.270484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:42.272035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:42.330895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015ddbb-1959-4841-942a-ea2e5bdb58fb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:42.340144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39022545-3451-46cf-a926-7a98e807b7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:42.340810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39022545-3451-46cf-a926-7a98e807b7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.341097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39022545-3451-46cf-a926-7a98e807b7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.341378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39022545-3451-46cf-a926-7a98e807b7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.351801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39022545-3451-46cf-a926-7a98e807b7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:42.353241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39022545-3451-46cf-a926-7a98e807b7b8 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:42.362602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1149/2300] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:42.632086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:42.633681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:42.645682 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629519e-2a7e-42ab-a399-5a099a584320 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:42.651004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:42.651865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.652151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.652430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] PATH_INFO: `/v3/projects/d7673fc376a940ee95c9381d83a5393f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.653639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7673fc376a940ee95c9381d83a5393f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:42.655624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:42.704768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7673fc376a940ee95c9381d83a5393f', 'request_id': 'req-d221c090-63b8-4878-8fa0-fe5886b3b51a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.711460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7673fc376a940ee95c9381d83a5393f', 'request_id': 'req-d221c090-63b8-4878-8fa0-fe5886b3b51a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.719057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7673fc376a940ee95c9381d83a5393f', 'request_id': 'req-d221c090-63b8-4878-8fa0-fe5886b3b51a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.722919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:42.732428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] The token cache is being invalidate because project d7673fc376a940ee95c9381d83a5393f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:42.732928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.733707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d221c090-63b8-4878-8fa0-fe5886b3b51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:42.735665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1152/2301] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:42 2022] DELETE /identity/v3/projects/d7673fc376a940ee95c9381d83a5393f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:42.740551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.741371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.741654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.760315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:42.761508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:42.821284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1150/2302] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:42.828926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.829266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.829585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.849130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:42.851238 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:42.859854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1153/2303] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:42.864054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:42.870219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:42.874034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d499e7-a34c-4803-82a3-fc0bcaeefeaa None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:42.879744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:42.880610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.881160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.881658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.882441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:10:42.882764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:42.884477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:42.896191 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1151/2304] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:10:42.901000 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:42.910391 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0938396-899b-4aba-a66f-35ec6b26f79a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:42.916101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:42.916800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:42.917089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:42.917371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:42.918071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:42.920042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:42.945420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1154/2305] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:42 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:42.949485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:42.960164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c849a-63ca-4896-89fb-302d4049d881 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.017442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.018156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.018447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.018732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.019470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.021405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.041769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1152/2306] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:10:42 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:43.045813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.056435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea50e7-9369-4afc-bbed-e07c3baf2654 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.061257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.061940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.062224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.062504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.063265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.064870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.076561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1155/2307] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:43.080344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.091224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deafc4f-14e7-4665-b2d2-b4c32a40906c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.096080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6285bb7-137b-4b55-8dd3-18999c253911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.096782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285bb7-137b-4b55-8dd3-18999c253911 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.097058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285bb7-137b-4b55-8dd3-18999c253911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.097384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285bb7-137b-4b55-8dd3-18999c253911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.098216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6285bb7-137b-4b55-8dd3-18999c253911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.099814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6285bb7-137b-4b55-8dd3-18999c253911 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.109634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1153/2308] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:43.113753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.123472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797a954-b5dd-4815-9cb3-d2a36f24aaad admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.128011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.128870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.129152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.129429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] PATH_INFO: `/v3/projects/550dc66ec75c49429146685dc62308c8/users/133ef233f1624e7fb94f4b6d56f1925f/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.144523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550dc66ec75c49429146685dc62308c8, user_id=133ef233f1624e7fb94f4b6d56f1925f, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.147940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.160619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1156/2309] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:43 2022] PUT /identity/v3/projects/550dc66ec75c49429146685dc62308c8/users/133ef233f1624e7fb94f4b6d56f1925f/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:43.170779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.171105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.171390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.198151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] MFA Rules not processed for user `133ef233f1624e7fb94f4b6d56f1925f`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:43.199403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:43.262430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1154/2310] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:43.266963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.328387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4829-bb8e-48f2-879e-3814a5ba5ac0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.333376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.334047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.334949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.335210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.336055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.337991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.367343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1157/2311] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:43 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:43.368553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.379366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02ee6-af59-4ec6-9904-a5ff3e4b00ae None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.437168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.437849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.438136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.438420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.439147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.441092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.448443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:43.464080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1155/2312] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:10:43 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:10:43.468737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.480284 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb824751-1ec4-4bd2-b8ae-539ff0fa0430 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.485452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.486119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.486409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.486691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.487457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.489097 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.500367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1158/2313] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:43.504445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.515139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ef736-3b62-48c3-bb32-46a52fdd8e30 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.519961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.520870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.521159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.521440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] PATH_INFO: `/v3/projects/edeae2febb584f749971d73ee4684be3/users/5e78d5caa6db4d8eb064db2d25cb8ade/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.536090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=edeae2febb584f749971d73ee4684be3, user_id=5e78d5caa6db4d8eb064db2d25cb8ade, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.539636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.552624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1156/2314] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:43 2022] PUT /identity/v3/projects/edeae2febb584f749971d73ee4684be3/users/5e78d5caa6db4d8eb064db2d25cb8ade/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:43.557477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.567624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805f513-bffe-401d-84ba-de13e31e02b1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.625875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2b3c-dd67-45f3-ba64-50f0fb2efd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.626621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2b3c-dd67-45f3-ba64-50f0fb2efd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.626924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2b3c-dd67-45f3-ba64-50f0fb2efd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.627209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2b3c-dd67-45f3-ba64-50f0fb2efd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.628133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2b3c-dd67-45f3-ba64-50f0fb2efd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.630074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2b3c-dd67-45f3-ba64-50f0fb2efd31 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.635976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:43.641426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1159/2315] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:43.645516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.655870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c47a75-de4e-478e-8b70-f4172d63dd6e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.660357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.661307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.661509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.661793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] PATH_INFO: `/v3/projects/edeae2febb584f749971d73ee4684be3/users/5e78d5caa6db4d8eb064db2d25cb8ade/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.664097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edeae2febb584f749971d73ee4684be3, user_id=5e78d5caa6db4d8eb064db2d25cb8ade, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.666736 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.676599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1157/2316] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:43 2022] PUT /identity/v3/projects/edeae2febb584f749971d73ee4684be3/users/5e78d5caa6db4d8eb064db2d25cb8ade/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:10:43.688675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.688967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.689246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.716049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] MFA Rules not processed for user `5e78d5caa6db4d8eb064db2d25cb8ade`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:43.717244 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:43.785137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1160/2317] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:43 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:43.797078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.798806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:43.878047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9a676-497e-40cf-96da-03b60cde13a4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.882053 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.882679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.883005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.883282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:43.892038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:43.893458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:43.903340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1158/2318] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:43.947808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:43.949216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:43.961922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe1eeb-008b-435e-b69c-89ee5ea04274 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:43.965760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba357ff-5baf-4262-96ac-1f577832a60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:43.966393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba357ff-5baf-4262-96ac-1f577832a60a service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:43.966676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba357ff-5baf-4262-96ac-1f577832a60a service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:43.967012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba357ff-5baf-4262-96ac-1f577832a60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:44.025024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba357ff-5baf-4262-96ac-1f577832a60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:44.026495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba357ff-5baf-4262-96ac-1f577832a60a service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:44.035443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1161/2319] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:47.350915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:47.419910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65578cb8-a06f-4b2f-8bae-3be5d895e4db service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:47.425172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:47.426050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:47.426335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:47.426609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] PATH_INFO: `/v3/users/133ef233f1624e7fb94f4b6d56f1925f` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:47.427877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] RBAC: Authorizing `identity:delete_user(user_id=133ef233f1624e7fb94f4b6d56f1925f)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:47.430517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:47.473799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:47.484150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133ef233f1624e7fb94f4b6d56f1925f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:47.485119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133ef233f1624e7fb94f4b6d56f1925f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:47.485743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133ef233f1624e7fb94f4b6d56f1925f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:47.493241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133ef233f1624e7fb94f4b6d56f1925f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:47.505596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133ef233f1624e7fb94f4b6d56f1925f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:47.519080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1159/2320] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:10:47 2022] DELETE /identity/v3/users/133ef233f1624e7fb94f4b6d56f1925f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:47.524416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:47.524873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:47.525014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:47.541741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:47.543038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:47.606068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1162/2321] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:47.616101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:47.617776 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:47.680605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484ef1e-fe76-40a8-b90d-650ac31f306c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:47.684344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:47.684987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:47.685272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:47.685553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:47.695204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:47.696658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:47.706526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1160/2322] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:47.982688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:47.984382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:47.996662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799064f-5336-4be9-988f-39bb5dc1213a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:48.001830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:48.002723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:48.003049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:48.003335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] PATH_INFO: `/v3/projects/550dc66ec75c49429146685dc62308c8` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:48.004726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550dc66ec75c49429146685dc62308c8)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:48.006890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:48.032009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:48.059567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550dc66ec75c49429146685dc62308c8', 'request_id': 'req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.065705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550dc66ec75c49429146685dc62308c8', 'request_id': 'req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.073494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550dc66ec75c49429146685dc62308c8', 'request_id': 'req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.085051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] The token cache is being invalidate because project 550dc66ec75c49429146685dc62308c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:48.085342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.085996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.087591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1163/2323] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:47 2022] DELETE /identity/v3/projects/550dc66ec75c49429146685dc62308c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:48.091914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:48.102595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca086e-72f8-4fb9-b4ad-b250bc85cefe service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:48.167935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:48.169943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:48.169943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:48.170345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] PATH_INFO: `/v3/users/5e78d5caa6db4d8eb064db2d25cb8ade` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:48.172163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e78d5caa6db4d8eb064db2d25cb8ade)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:48.174332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:48.235244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e78d5caa6db4d8eb064db2d25cb8ade'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.236078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e78d5caa6db4d8eb064db2d25cb8ade'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.237184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e78d5caa6db4d8eb064db2d25cb8ade'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.248613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e78d5caa6db4d8eb064db2d25cb8ade'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.251111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:48.264682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b418e570-a779-416d-ae80-9b18338ad95c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e78d5caa6db4d8eb064db2d25cb8ade'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.280201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1161/2324] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:10:48 2022] DELETE /identity/v3/users/5e78d5caa6db4d8eb064db2d25cb8ade => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:48.669306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:48.670930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:48.687932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e3196-8d7b-41f1-91d9-9e6d8efb9c15 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:48.753719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:48.755014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:48.755374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:48.755755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] PATH_INFO: `/v3/projects/edeae2febb584f749971d73ee4684be3` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:48.757107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edeae2febb584f749971d73ee4684be3)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:48.759089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:48.814191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edeae2febb584f749971d73ee4684be3', 'request_id': 'req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.821348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edeae2febb584f749971d73ee4684be3', 'request_id': 'req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.829040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edeae2febb584f749971d73ee4684be3', 'request_id': 'req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.842804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] The token cache is being invalidate because project edeae2febb584f749971d73ee4684be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:48.843139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.843868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-bbffc1e6-922e-4c43-a0b5-0789b2f615a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:48.846275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1164/2325] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:48 2022] DELETE /identity/v3/projects/edeae2febb584f749971d73ee4684be3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:48.851782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:48.852292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:48.852452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:48.870349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:48.871584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:48.933033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1162/2326] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:48.940765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:48.941059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:48.941349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:48.961770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:48.963038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:48.971841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1165/2327] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:48.976699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:48.986260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cf965-47e6-4042-bdb9-11b58c662164 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:48.991844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:48.993056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:48.993673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:48.994291 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:48.995120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:10:48.995501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:48.997180 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.011051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1163/2328] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:10:49.015474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.025169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae60c9-32ab-4672-b056-3daa7aa65a0a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.029997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.030731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.031130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.031482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.032267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.034289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.064953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1166/2329] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:49 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:49.066184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.076338 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2a01-2609-476f-a272-cb201b91a192 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.135784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.136561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.136914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.137270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.138034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.140032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.163400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1164/2330] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:10:49 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:49.167538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.178936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd826c-86b5-42bf-ade0-1aae445ca03a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.183774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.184435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.184721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.185002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.185723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.187371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.189533 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:49.199884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1167/2331] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:49.203967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.216430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12c11-997a-4431-a339-d9caf4e240cd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.223019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffd0b2-25e0-4a23-b9a6-32395dc5746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.223772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffd0b2-25e0-4a23-b9a6-32395dc5746d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.224130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffd0b2-25e0-4a23-b9a6-32395dc5746d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.224478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffd0b2-25e0-4a23-b9a6-32395dc5746d admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.225875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffd0b2-25e0-4a23-b9a6-32395dc5746d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.228712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffd0b2-25e0-4a23-b9a6-32395dc5746d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.246540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1165/2332] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:49.248157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.260855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3c2a6-8c60-4b9b-aa1d-7208fdd522af admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.266103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.267197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.267483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.267763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] PATH_INFO: `/v3/projects/daec4accff19472ebcdb28dd909a2c04/users/4b1c4506b1b14de99e95575660618952/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.286289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daec4accff19472ebcdb28dd909a2c04, user_id=4b1c4506b1b14de99e95575660618952, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.289992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.307803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1168/2333] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:49 2022] PUT /identity/v3/projects/daec4accff19472ebcdb28dd909a2c04/users/4b1c4506b1b14de99e95575660618952/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:49.319281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.319576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.319866 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.351549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] MFA Rules not processed for user `4b1c4506b1b14de99e95575660618952`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:49.353474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:49.391367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:49.434247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1166/2334] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:49.438386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.520172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb823b-f863-494d-9ee7-fc6116eb6fb9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.525657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.526336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.526624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.526946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.527642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.529601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.566883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.570936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1169/2335] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:49 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:49.577537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914fd4-b3a1-48eb-a277-cf3add0f383c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.645249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.646131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.646442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.646761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.647506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.649462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.672653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1167/2336] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:10:49 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:49.676577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.687990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1198f4-f34e-429e-95be-d88165b86fe1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.692691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.693382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.694253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.694532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.695292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.696931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.707891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1170/2337] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:49.712253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.723054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8def-4c8c-47fd-86d0-5715c0b9793b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.729023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.730107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.730433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.730979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] PATH_INFO: `/v3/projects/a2f58f2ce6e34866925d16c229beae9e/users/47ee8a330dcf41fbaf84910bdff499c6/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.748512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f58f2ce6e34866925d16c229beae9e, user_id=47ee8a330dcf41fbaf84910bdff499c6, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.751433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.773045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1168/2338] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:49 2022] PUT /identity/v3/projects/a2f58f2ce6e34866925d16c229beae9e/users/47ee8a330dcf41fbaf84910bdff499c6/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:49.777423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.799237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3e0ca-90e0-4063-b21d-3a3aeba98549 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.880048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bdc34-e4be-4ad8-b89e-f323a06d0f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.880636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bdc34-e4be-4ad8-b89e-f323a06d0f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.881017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bdc34-e4be-4ad8-b89e-f323a06d0f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.881273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bdc34-e4be-4ad8-b89e-f323a06d0f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.882770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bdc34-e4be-4ad8-b89e-f323a06d0f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.884459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bdc34-e4be-4ad8-b89e-f323a06d0f50 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.898802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1171/2339] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:49.900036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:49.911087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4a74d-c68c-495a-9a32-c488fc2e910f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:49.916846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:49.917838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.918190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.918560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] PATH_INFO: `/v3/projects/a2f58f2ce6e34866925d16c229beae9e/users/47ee8a330dcf41fbaf84910bdff499c6/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.920772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f58f2ce6e34866925d16c229beae9e, user_id=47ee8a330dcf41fbaf84910bdff499c6, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:49.923663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:49.945205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1169/2340] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:49 2022] PUT /identity/v3/projects/a2f58f2ce6e34866925d16c229beae9e/users/47ee8a330dcf41fbaf84910bdff499c6/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:49.947366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:49.947735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:49.948094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:49.977446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] MFA Rules not processed for user `47ee8a330dcf41fbaf84910bdff499c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:49.978675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:50.013939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:50.053552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1172/2341] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:50.065696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:50.067439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:50.140160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f196dc7-6443-42b2-91a7-e1f945d156c1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:50.144240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:50.145331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:50.146072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:50.146633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:50.162430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:50.162430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:50.171993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1170/2342] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:50.540744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:50.551160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3fec1-e4a3-4315-8c80-8f8c1ab0318e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:50.565378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47ee8a330dcf41fbaf84910bdff499c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2f58f2ce6e34866925d16c229beae9e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:50.566174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:50.566535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:50.566920 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:50.567788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:10:50.568071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:50.579578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:50.595589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1173/2343] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:10:50.600319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:50.610478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386350b-8374-4022-8356-2520a1307586 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:50.617492 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47ee8a330dcf41fbaf84910bdff499c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2f58f2ce6e34866925d16c229beae9e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:50.618254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:50.618483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:50.618767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:50.619548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:50.621530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:50.654960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1171/2344] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:50 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:10:50.658223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:50.669072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a7f27-e9c2-4c0c-9a3f-30f87a4ff85a tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:50.742643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47ee8a330dcf41fbaf84910bdff499c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2f58f2ce6e34866925d16c229beae9e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:50.747308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:50.747693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:50.748056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:50.748905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:10:50.749283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:50.751080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:50.768520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1174/2345] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:10:50.811690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:50.819285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:50.823561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d7790-0298-4b88-8a7d-a6344fe63e03 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:50.832928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47ee8a330dcf41fbaf84910bdff499c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2f58f2ce6e34866925d16c229beae9e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:50.835344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:50.836163 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:50.836825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] PATH_INFO: `/v3/projects/13df4f84c56f4b42b8750d9c0d6aaca3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:50.838680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorizing `identity:delete_project(project_id=13df4f84c56f4b42b8750d9c0d6aaca3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:50.840735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:50.894301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13df4f84c56f4b42b8750d9c0d6aaca3', 'request_id': 'req-a5627258-c257-4334-b89e-ce7719cb31e2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:50.900248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13df4f84c56f4b42b8750d9c0d6aaca3', 'request_id': 'req-a5627258-c257-4334-b89e-ce7719cb31e2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:50.906450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13df4f84c56f4b42b8750d9c0d6aaca3', 'request_id': 'req-a5627258-c257-4334-b89e-ce7719cb31e2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:50.919779 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] The token cache is being invalidate because project 13df4f84c56f4b42b8750d9c0d6aaca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:50.920057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:50.920724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a5627258-c257-4334-b89e-ce7719cb31e2 tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:50.922400 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1172/2346] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:50 2022] DELETE /identity/v3/projects/13df4f84c56f4b42b8750d9c0d6aaca3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:51.121808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:51.123982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:51.200784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b48ca-a149-47bf-a223-a67d5fde3a0b tempest-VolumeQuotaClassesTest-2030193372 tempest-VolumeQuotaClassesTest-2030193372-project-admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:51.253251 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:51.260168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:51.261049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:51.261330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:51.261604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] PATH_INFO: `/v3/users/4b1c4506b1b14de99e95575660618952` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:51.262773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1c4506b1b14de99e95575660618952)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:51.264750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:51.309832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1c4506b1b14de99e95575660618952'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.310467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1c4506b1b14de99e95575660618952'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.317152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1c4506b1b14de99e95575660618952'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.317774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1c4506b1b14de99e95575660618952'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.326008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1c4506b1b14de99e95575660618952'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.343215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1175/2347] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:10:51 2022] DELETE /identity/v3/users/4b1c4506b1b14de99e95575660618952 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:10:51.347707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:51.348012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:51.348315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:51.367777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:51.368979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:51.432764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1173/2348] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:51.442004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:51.443476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:51.509784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1492136-a471-46c4-861c-6d88560eddc6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:51.513555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:51.514405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:51.514687 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:51.514997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:51.526024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:51.527598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:51.538953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1176/2349] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:10:51.830047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:51.831671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:51.846071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d1c49-b6a8-437e-9de2-e2319d602a01 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:51.851078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:51.851816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:51.852983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:51.852983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] PATH_INFO: `/v3/projects/daec4accff19472ebcdb28dd909a2c04` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:51.854100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daec4accff19472ebcdb28dd909a2c04)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:51.856465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:51.908610 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daec4accff19472ebcdb28dd909a2c04', 'request_id': 'req-24181d8f-719a-4258-9efe-d499ff6a5ff9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.915042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daec4accff19472ebcdb28dd909a2c04', 'request_id': 'req-24181d8f-719a-4258-9efe-d499ff6a5ff9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.921225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daec4accff19472ebcdb28dd909a2c04', 'request_id': 'req-24181d8f-719a-4258-9efe-d499ff6a5ff9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.934389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] The token cache is being invalidate because project daec4accff19472ebcdb28dd909a2c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:51.934677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.935337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-24181d8f-719a-4258-9efe-d499ff6a5ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:51.936879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1174/2350] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:51 2022] DELETE /identity/v3/projects/daec4accff19472ebcdb28dd909a2c04 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:51.942862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:51.954651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f28534-f7e4-474f-93cb-bf7880aa0ee6 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:51.994890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:51.995371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:51.995681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.025493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] MFA Rules not processed for user `adb92379209e46e79de4df370183fa8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:52.026832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:52.028700 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:52.029603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.029905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.030182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] PATH_INFO: `/v3/users/47ee8a330dcf41fbaf84910bdff499c6` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.031707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ee8a330dcf41fbaf84910bdff499c6)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:52.032695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:52.043014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:52.058602 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:52.086948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ee8a330dcf41fbaf84910bdff499c6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.087636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ee8a330dcf41fbaf84910bdff499c6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.092950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1175/2351] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:10:52.098926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ee8a330dcf41fbaf84910bdff499c6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.099626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ee8a330dcf41fbaf84910bdff499c6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.109276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-92e9d2df-f1f7-4aae-82cc-55d819ecfe54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ee8a330dcf41fbaf84910bdff499c6'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.127837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1177/2352] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:10:51 2022] DELETE /identity/v3/users/47ee8a330dcf41fbaf84910bdff499c6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:52.426684 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:52.428913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:52.449576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabb3a-cec5-49a3-867d-b21fb4242405 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:52.513262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:52.514299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.514651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.515037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] PATH_INFO: `/v3/projects/a2f58f2ce6e34866925d16c229beae9e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.516411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f58f2ce6e34866925d16c229beae9e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:52.518576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:52.567198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f58f2ce6e34866925d16c229beae9e', 'request_id': 'req-040a0786-b09e-4da6-88e3-81e681057e07'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.573367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f58f2ce6e34866925d16c229beae9e', 'request_id': 'req-040a0786-b09e-4da6-88e3-81e681057e07'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.579438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f58f2ce6e34866925d16c229beae9e', 'request_id': 'req-040a0786-b09e-4da6-88e3-81e681057e07'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.594072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] The token cache is being invalidate because project a2f58f2ce6e34866925d16c229beae9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:10:52.594361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.595026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-040a0786-b09e-4da6-88e3-81e681057e07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:10:52.596597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1176/2353] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:10:52 2022] DELETE /identity/v3/projects/a2f58f2ce6e34866925d16c229beae9e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:52.603186 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.603554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.603908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.621528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:52.622744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:52.686892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1178/2354] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:52.694196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.694570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.694883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.715042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:52.716258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:52.728667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1177/2355] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:52.729840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:52.740271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd1e39-862b-4ef5-8989-6eee20eef47e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:52.744719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:52.745377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.745660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.745935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.746659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:10:52.747006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:52.748622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:52.761212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1179/2356] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:10:52.766046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:52.775819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615879e-25c6-45ce-b3b6-73aad9481bd3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:52.780552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:52.781225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.781496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.781774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.782471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:52.784444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:52.815201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1178/2357] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:10:52 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:52.816398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:52.827209 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aad83-89aa-40ea-af5e-859f1c4f2499 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:52.917516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:52.918919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.919389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.919886 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.921505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:52.924744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:52.952455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1180/2358] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:10:52 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:10:52.966138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:52.979712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095bedc-f66d-47a1-91e3-c67b1a472cdd admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:52.984968 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:52.986397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:52.986723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:52.987057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:52.987797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:52.989553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:53.001144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1179/2359] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:53.004943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:53.021789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad080b-c59c-4e53-9852-1fa10f3c6009 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:53.029067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0c1f2-e1b6-4597-9b79-06561d5fa198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:53.029958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0c1f2-e1b6-4597-9b79-06561d5fa198 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:53.030443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0c1f2-e1b6-4597-9b79-06561d5fa198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:53.030938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0c1f2-e1b6-4597-9b79-06561d5fa198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:53.031975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0c1f2-e1b6-4597-9b79-06561d5fa198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:53.034906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0c1f2-e1b6-4597-9b79-06561d5fa198 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:53.051745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1181/2360] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:10:53.056557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:53.066612 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956692cc-78ec-4947-b5f5-2a51be715593 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:53.071826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:53.072737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:53.073024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:53.073306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] PATH_INFO: `/v3/projects/b5af4c91e46245449a11637bf3af7342/users/9f5f50394a8a46f59e77b5945fc84182/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:53.075753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:53.090981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5af4c91e46245449a11637bf3af7342, user_id=9f5f50394a8a46f59e77b5945fc84182, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:53.094571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:53.118721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1180/2361] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:10:53 2022] PUT /identity/v3/projects/b5af4c91e46245449a11637bf3af7342/users/9f5f50394a8a46f59e77b5945fc84182/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:10:53.130200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048228fc-a603-4186-8fec-46b5a17f6749 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:53.130500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048228fc-a603-4186-8fec-46b5a17f6749 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:53.130882 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048228fc-a603-4186-8fec-46b5a17f6749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:53.140098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:10:53.163288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-048228fc-a603-4186-8fec-46b5a17f6749 None None] MFA Rules not processed for user `9f5f50394a8a46f59e77b5945fc84182`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:10:53.164761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-048228fc-a603-4186-8fec-46b5a17f6749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:53.235194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1182/2362] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:10:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:10:53.247141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:10:53.249129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:10:53.311766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a69bfb-e786-45f6-8fbd-ecaf664b0f6d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:10:53.315740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:10:53.316403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:10:53.316817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:10:53.317112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:10:53.326751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:10:53.328254 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:10:53.337827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1181/2363] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:10:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:11:02.792276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212a3e8-ae1f-4ab7-869d-4114d700bcef None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:02.792873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212a3e8-ae1f-4ab7-869d-4114d700bcef None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:02.792873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212a3e8-ae1f-4ab7-869d-4114d700bcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:02.811035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d212a3e8-ae1f-4ab7-869d-4114d700bcef None None] MFA Rules not processed for user `6c904a413f6a4ba49686debac817c70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:02.812237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d212a3e8-ae1f-4ab7-869d-4114d700bcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:02.875202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1183/2364] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:11:03.333112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:12.868422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=55 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:41.285273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:41.347566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb9357-0d74-4c23-8d73-5f41736f915a service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:41.352143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:41.353117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:41.353443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:41.353744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] PATH_INFO: `/v3/users/9f5f50394a8a46f59e77b5945fc84182` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:41.355036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5f50394a8a46f59e77b5945fc84182)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:41.357636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:41.401870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5f50394a8a46f59e77b5945fc84182'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.402521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5f50394a8a46f59e77b5945fc84182'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.403211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5f50394a8a46f59e77b5945fc84182'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.410399 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5f50394a8a46f59e77b5945fc84182'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.423074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5f50394a8a46f59e77b5945fc84182'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.446883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1182/2365] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:11:41 2022] DELETE /identity/v3/users/9f5f50394a8a46f59e77b5945fc84182 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:11:41.451367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:41.451873 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:41.451974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:41.469334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:41.470523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:41.527622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1184/2366] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:11:41.537226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:41.538621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:41.597943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cb5ee-24fb-431e-9ace-da519a880743 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:41.611726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d87f4-29db-48a4-b91f-851df1d2b766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:41.612397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d87f4-29db-48a4-b91f-851df1d2b766 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:41.612682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d87f4-29db-48a4-b91f-851df1d2b766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:41.612961 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d87f4-29db-48a4-b91f-851df1d2b766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:41.623314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d87f4-29db-48a4-b91f-851df1d2b766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:41.624744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d87f4-29db-48a4-b91f-851df1d2b766 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:41.634355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1183/2367] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:11:41.905151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:41.906656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:41.919333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a6cc9-4b7e-461c-b016-fea642c9781e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:41.924824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:41.925696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:41.925990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:41.926307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] PATH_INFO: `/v3/projects/b5af4c91e46245449a11637bf3af7342` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:41.928572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5af4c91e46245449a11637bf3af7342)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:41.930559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:41.978132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5af4c91e46245449a11637bf3af7342', 'request_id': 'req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.985311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5af4c91e46245449a11637bf3af7342', 'request_id': 'req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:41.993922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5af4c91e46245449a11637bf3af7342', 'request_id': 'req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:42.005689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] The token cache is being invalidate because project b5af4c91e46245449a11637bf3af7342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:11:42.005990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:42.006635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-74b3b23b-5ef7-4fd8-859d-589cedd81ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:42.008235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1185/2368] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:11:41 2022] DELETE /identity/v3/projects/b5af4c91e46245449a11637bf3af7342 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:11:42.013888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.014188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.014471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.032716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:42.033854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:42.050285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:42.095657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1184/2369] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:11:42.103438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.103740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.104025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.123706 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:42.124912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:42.132951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1186/2370] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:11:42.137011 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.147309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f1892-df00-441c-90c7-f6e5ab87361a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.151787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.152426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.152709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.152983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.154417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:11:42.154983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.156597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.167861 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1185/2371] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:11:42.172048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.181363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8c967-b1ff-405b-bd45-fe0c4732e3ca None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.186194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.186859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.187156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.187440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.188132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.190067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.215829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1187/2372] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:11:42 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:11:42.220659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.230440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfebb-6f36-4823-9054-baec74c6f85d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.285804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.286409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.286654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.286960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.287651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.289561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.309201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1186/2373] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:11:42 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:11:42.313548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.323797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685605c-5a42-4ad5-ad87-f678af55a56d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.329126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.329780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.330070 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.330356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.331119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.332738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.343982 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1188/2374] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:11:42.347808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.358264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd009d91-2655-4657-8b27-b90988d8f0f5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.362781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34791b-891f-4ede-821c-9bbfb114419a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.363462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34791b-891f-4ede-821c-9bbfb114419a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.363754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34791b-891f-4ede-821c-9bbfb114419a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.364035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34791b-891f-4ede-821c-9bbfb114419a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.364755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34791b-891f-4ede-821c-9bbfb114419a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.366344 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34791b-891f-4ede-821c-9bbfb114419a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.378150 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1187/2375] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:11:42.382204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.392714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985db3-62b7-4e65-a11f-0bce62f44c4e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.397231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.398089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.398398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.398676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] PATH_INFO: `/v3/projects/ab580f130a82421fa767b47aa05a4331/users/c28e133ac6cb49da91b6a325685d83f9/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.413046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab580f130a82421fa767b47aa05a4331, user_id=c28e133ac6cb49da91b6a325685d83f9, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.415777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.428993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1189/2376] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:11:42 2022] PUT /identity/v3/projects/ab580f130a82421fa767b47aa05a4331/users/c28e133ac6cb49da91b6a325685d83f9/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:11:42.441240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.441537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.441834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.469943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] MFA Rules not processed for user `c28e133ac6cb49da91b6a325685d83f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:42.471225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:42.535973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1188/2377] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:11:42.547095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:42.548856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:42.570849 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:42.608719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3fca5-31b7-4301-8d29-6366f80e410a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:42.612487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7271e-c197-41fb-8431-a55c8680aef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:42.613140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7271e-c197-41fb-8431-a55c8680aef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:42.613429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7271e-c197-41fb-8431-a55c8680aef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:42.613710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7271e-c197-41fb-8431-a55c8680aef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:42.622044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7271e-c197-41fb-8431-a55c8680aef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:42.623606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7271e-c197-41fb-8431-a55c8680aef7 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:42.632703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1190/2378] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:11:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:11:45.155078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:45.174173 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:45.217765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2499f-5d8e-47d8-9dac-644ddcbd1319 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:45.222247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:45.223154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:45.223442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:45.223725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] PATH_INFO: `/v3/users/c28e133ac6cb49da91b6a325685d83f9` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:45.224923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28e133ac6cb49da91b6a325685d83f9)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:45.226878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:45.271828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28e133ac6cb49da91b6a325685d83f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.272413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28e133ac6cb49da91b6a325685d83f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.273017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28e133ac6cb49da91b6a325685d83f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.279986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28e133ac6cb49da91b6a325685d83f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.291601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28e133ac6cb49da91b6a325685d83f9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.305185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1189/2379] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:11:45 2022] DELETE /identity/v3/users/c28e133ac6cb49da91b6a325685d83f9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:11:45.309646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:45.310922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:45.311127 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:45.327937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:45.329137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:45.387298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1191/2380] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:11:45.397186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:45.398563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:45.458324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3813e2-1076-4447-b50f-c029867c236a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:45.461905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb3532-473f-4724-bcca-3b5aae03a7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:45.462544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3532-473f-4724-bcca-3b5aae03a7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:45.462830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3532-473f-4724-bcca-3b5aae03a7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:45.463167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3532-473f-4724-bcca-3b5aae03a7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:45.473238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb3532-473f-4724-bcca-3b5aae03a7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:45.474667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb3532-473f-4724-bcca-3b5aae03a7b1 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:45.484707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1190/2381] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:11:45.751623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:45.753116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:45.767784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f9670-f272-46f7-a6a1-94be973a5c32 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:45.772444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:45.773356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:45.773646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:45.774135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] PATH_INFO: `/v3/projects/ab580f130a82421fa767b47aa05a4331` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:45.775368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab580f130a82421fa767b47aa05a4331)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:45.777329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:45.826061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab580f130a82421fa767b47aa05a4331', 'request_id': 'req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.832202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab580f130a82421fa767b47aa05a4331', 'request_id': 'req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.839386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab580f130a82421fa767b47aa05a4331', 'request_id': 'req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.852443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] The token cache is being invalidate because project ab580f130a82421fa767b47aa05a4331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:11:45.852782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.853640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b13cb9b2-7b1f-4e0c-bba6-7b2c8b158459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:11:45.855516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1192/2382] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:11:45 2022] DELETE /identity/v3/projects/ab580f130a82421fa767b47aa05a4331 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:11:45.861147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:45.861446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:45.861734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:45.879851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:45.881050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:45.913429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:45.941504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1191/2383] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:11:45.949206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:45.949516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:45.949812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:45.971109 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:45.972349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:45.980837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1193/2384] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:11:45.985628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:45.995269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad8150-a2df-4389-ac0d-961325fb8077 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.001377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.002118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.002407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.002685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.003447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:11:46.003758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.005370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.022395 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1192/2385] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:11:46.023541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:46.032909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b566d7-389f-483b-941f-ed37946e8188 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.037572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.038212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.038496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.038774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.039485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.041385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.069296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1194/2386] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:11:46 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:11:46.073452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:46.084202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bfe6a-ab19-4bf7-adff-974174b99a92 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.148962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.149731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.150026 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.150309 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.151036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.153237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.175253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1193/2387] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:11:46 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:11:46.179351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:46.192053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abb367-6f4f-4656-bdcd-429bcbea3eb0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.198286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.199091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.199288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.199545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.200265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.202435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.215268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1195/2388] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:11:46.219770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:46.235771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fc99c-129e-4af9-afbd-8ae8acd4ce14 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.240276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65a200-87b9-4852-93ff-96888e8eb9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.241027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65a200-87b9-4852-93ff-96888e8eb9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.241230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65a200-87b9-4852-93ff-96888e8eb9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.241430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65a200-87b9-4852-93ff-96888e8eb9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.242154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65a200-87b9-4852-93ff-96888e8eb9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.243775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65a200-87b9-4852-93ff-96888e8eb9aa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.256201 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1194/2389] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:11:46.260221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:46.271572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e409-6674-41c8-8c4c-c52654483e2f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.276310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.277198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.277487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.277795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] PATH_INFO: `/v3/projects/e73610b6b8854229a7cec01d981ac8f2/users/d9b211030ac843c7beb1d5c6382202ed/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.280304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:46.301010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73610b6b8854229a7cec01d981ac8f2, user_id=d9b211030ac843c7beb1d5c6382202ed, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.303769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.319390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1196/2390] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:11:46 2022] PUT /identity/v3/projects/e73610b6b8854229a7cec01d981ac8f2/users/d9b211030ac843c7beb1d5c6382202ed/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:11:46.330249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.330544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.330825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.358721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] MFA Rules not processed for user `d9b211030ac843c7beb1d5c6382202ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:11:46.359894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:46.427496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1195/2391] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:11:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:11:46.444479 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:11:46.446262 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:11:46.502090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274c462-a78f-4d3c-83fd-75f350e60578 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:11:46.505728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c99366-7d7a-4746-8e02-d1f453bdc18c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:11:46.506364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c99366-7d7a-4746-8e02-d1f453bdc18c service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:11:46.506648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c99366-7d7a-4746-8e02-d1f453bdc18c service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:11:46.506961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c99366-7d7a-4746-8e02-d1f453bdc18c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:11:46.515911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c99366-7d7a-4746-8e02-d1f453bdc18c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:11:46.517378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c99366-7d7a-4746-8e02-d1f453bdc18c service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:11:46.526218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1197/2392] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:11:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:11:56.424479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:11:56.522248 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:12:08.805480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:12:08.892164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a47302-94eb-4577-99c8-a428672591ba None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:12:08.897623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:12:08.898613 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:08.899038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:08.899557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] PATH_INFO: `/v3/users/d9b211030ac843c7beb1d5c6382202ed` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:08.901014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b211030ac843c7beb1d5c6382202ed)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:12:08.903105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:12:08.959222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b211030ac843c7beb1d5c6382202ed'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:08.960045 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b211030ac843c7beb1d5c6382202ed'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:08.960751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b211030ac843c7beb1d5c6382202ed'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:08.969926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b211030ac843c7beb1d5c6382202ed'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:08.983129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b211030ac843c7beb1d5c6382202ed'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:08.996742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1196/2393] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:12:08 2022] DELETE /identity/v3/users/d9b211030ac843c7beb1d5c6382202ed => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:12:09.001551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:09.002074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:09.002409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:09.022033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:12:09.023396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:09.091553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1198/2394] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:12:09.101746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:12:09.103246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:09.163572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171369a4-f256-45cb-9190-6df0985b25cf admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:12:09.167324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dcd61-f08d-4d3d-a62e-c56fd9259d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:12:09.167979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dcd61-f08d-4d3d-a62e-c56fd9259d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:09.168283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dcd61-f08d-4d3d-a62e-c56fd9259d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:09.168556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dcd61-f08d-4d3d-a62e-c56fd9259d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:09.179263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dcd61-f08d-4d3d-a62e-c56fd9259d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:12:09.180707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dcd61-f08d-4d3d-a62e-c56fd9259d11 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:12:09.191561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1197/2395] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:12:09.487021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:12:09.488515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:09.501915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c8f02-3105-41e2-a05e-2131680afbdb None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:12:09.506928 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:12:09.508016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:09.508503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:09.508860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] PATH_INFO: `/v3/projects/e73610b6b8854229a7cec01d981ac8f2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:09.510261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73610b6b8854229a7cec01d981ac8f2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:12:09.512399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:12:09.565901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73610b6b8854229a7cec01d981ac8f2', 'request_id': 'req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:09.572080 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73610b6b8854229a7cec01d981ac8f2', 'request_id': 'req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:09.580138 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73610b6b8854229a7cec01d981ac8f2', 'request_id': 'req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:09.590611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] The token cache is being invalidate because project e73610b6b8854229a7cec01d981ac8f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:12:09.590932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:09.591560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4fe8ce92-6648-45ea-83f1-bedb3eb6ebf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:12:09.593119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1199/2396] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:12:09 2022] DELETE /identity/v3/projects/e73610b6b8854229a7cec01d981ac8f2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:12:11.969682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:11.970184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:11.970290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:11.987497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] MFA Rules not processed for user `6aebb057a324492699141a878793c097`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:12:11.988715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:12.005283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:12:12.055433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1198/2397] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:12:19.588015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:12:22.048460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=15 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:12:42.185980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:42.186496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:42.186671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:42.206459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:12:42.207707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:42.262568 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1200/2398] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:12:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:12:42.308597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:12:42.310041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:42.382911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14370a-73fd-45cb-881c-26372be34644 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:12:42.387774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e90960-a158-447f-97ee-83f09ce6f0ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:12:42.388421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e90960-a158-447f-97ee-83f09ce6f0ab service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:42.388706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e90960-a158-447f-97ee-83f09ce6f0ab service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:42.388983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e90960-a158-447f-97ee-83f09ce6f0ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:42.398019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e90960-a158-447f-97ee-83f09ce6f0ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:12:42.399651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e90960-a158-447f-97ee-83f09ce6f0ab service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:12:42.407917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1199/2399] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:12:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:12:46.800618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:12:46.802036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:46.881455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61f3c-be2e-4bdd-8497-1476d35422a0 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:12:46.886992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:12:46.887767 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:46.888116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:46.888400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:46.890302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:46.901923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:12:46.903561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dd20a-5716-4554-9c85-ab4b15f44c17 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:12:46.914259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1201/2400] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:12:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:12:52.404301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=32 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:12:56.909820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=57 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:12:58.902993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555138-eee3-44f3-899c-c89be4b17c1c None None] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:58.903539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555138-eee3-44f3-899c-c89be4b17c1c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:58.903539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555138-eee3-44f3-899c-c89be4b17c1c None None] PATH_INFO: `/` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:58.904928 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1200/2401] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 12:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 12:12:58.909990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e999c4-dd49-49cc-b8a1-0441941035cc None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:58.910411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e999c4-dd49-49cc-b8a1-0441941035cc None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:58.910549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e999c4-dd49-49cc-b8a1-0441941035cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:58.928646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b8e999c4-dd49-49cc-b8a1-0441941035cc None None] MFA Rules not processed for user `b0208c5889a74870b764c7eb954242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:12:58.930020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b8e999c4-dd49-49cc-b8a1-0441941035cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:58.998881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1202/2402] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:12:59.010345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20555138-eee3-44f3-899c-c89be4b17c1c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:12:59.012084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-20555138-eee3-44f3-899c-c89be4b17c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:12:59.073763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20555138-eee3-44f3-899c-c89be4b17c1c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:12:59.077239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:12:59.077999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:12:59.078505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:12:59.078908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:12:59.089851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:12:59.091570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:12:59.102141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1201/2403] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:12:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:00.783163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:00.783701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:00.783951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:00.802271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:13:00.803481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:00.810750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1203/2404] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:13:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:13:00.822890 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:00.833872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef211-bbb3-4c04-a861-9caef5ce2de8 service glance] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:00.838747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:00.839397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:00.839681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:00.839958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:00.848517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:00.850083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:00.858089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1202/2405] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:13:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 12:13:02.293170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:02.294577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:02.351113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e99c3-bb5a-4807-b823-74ed699e2906 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:02.354669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:02.355344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:02.355631 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:02.355915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:02.357672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:02.424390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:02.425870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a785cc-8a97-44f2-9fd2-dc0444939945 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:02.435877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1204/2406] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:10.854934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=32 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:12.431921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=82 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:21.898786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:21.911029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d494411-849f-4b04-bdc1-0283498d19d9 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:21.914908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:21.915558 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:21.915848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:21.916208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:21.918036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:21.997141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:21.998765 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:22.010998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1203/2407] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:24.253963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:24.254447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:24.254556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:24.273392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:13:24.274600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:24.282321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1205/2408] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 12:13:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:13:24.295924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:24.307384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9c9c0-a7d7-4215-896f-18131369b93d service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:24.312553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:24.313383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:24.313675 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:24.313986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:24.322875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:24.325020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:24.336144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1204/2409] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:13:25.477180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:25.478755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:25.495045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c4ac-f2e0-4645-b860-c536dcc65572 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:25.499986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:25.500627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:25.500914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:25.501196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:25.503037 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:25.515812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:25.518053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:25.530139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1206/2410] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:28.624250 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:28.626720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:28.648365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa5d33-e241-4f76-b97a-3f871e8ac0e2 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:28.653304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:28.654314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:28.654670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:28.655059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] PATH_INFO: `/v3/users/567389c79da54aec809e004e4bde99e5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:28.656306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=567389c79da54aec809e004e4bde99e5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:28.658529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:28.710706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567389c79da54aec809e004e4bde99e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:28.711476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567389c79da54aec809e004e4bde99e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:28.712195 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567389c79da54aec809e004e4bde99e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:28.722566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567389c79da54aec809e004e4bde99e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:28.739110 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567389c79da54aec809e004e4bde99e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:28.754113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1205/2411] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:13:28 2022] DELETE /identity/v3/users/567389c79da54aec809e004e4bde99e5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:13:28.936569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:28.949914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db15501-53ea-4095-917a-39ca6281b64e service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:29.025335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:29.026317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:29.026677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:29.027055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] PATH_INFO: `/v3/projects/43703500ab064d7091826ea7870273fb` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:29.028448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43703500ab064d7091826ea7870273fb)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:29.030467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:29.097665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43703500ab064d7091826ea7870273fb', 'request_id': 'req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.104938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43703500ab064d7091826ea7870273fb', 'request_id': 'req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.112595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43703500ab064d7091826ea7870273fb', 'request_id': 'req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.124832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] The token cache is being invalidate because project 43703500ab064d7091826ea7870273fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:13:29.125221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.125866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.127473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1207/2412] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:13:28 2022] DELETE /identity/v3/projects/43703500ab064d7091826ea7870273fb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:13:29.131608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:29.133166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:29.147653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436ec25-007c-4948-abf0-dc3bcdc41ca4 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:29.203238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:29.224814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:29.225825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:29.226179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:29.226532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] PATH_INFO: `/v3/users/7ff3b7de582749369390300f57fd7503` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:29.227847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff3b7de582749369390300f57fd7503)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:29.229901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:29.281373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff3b7de582749369390300f57fd7503'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.282037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff3b7de582749369390300f57fd7503'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.282654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff3b7de582749369390300f57fd7503'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.290695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff3b7de582749369390300f57fd7503'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.303883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff3b7de582749369390300f57fd7503'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.316687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1206/2413] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:13:29 2022] DELETE /identity/v3/users/7ff3b7de582749369390300f57fd7503 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:13:29.506896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:29.508372 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:29.522531 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c13f5-e8ce-49b8-ac65-905c0a5f41a3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:29.580009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:29.580908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:29.581201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:29.581483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] PATH_INFO: `/v3/projects/c2bad9bd51b247e78dbf42c6c6ab4528` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:29.582659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2bad9bd51b247e78dbf42c6c6ab4528)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:29.585139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:29.597115 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:29.632932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2bad9bd51b247e78dbf42c6c6ab4528', 'request_id': 'req-9bcb29b2-e321-4479-a10e-bb0285ad3660'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.639245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2bad9bd51b247e78dbf42c6c6ab4528', 'request_id': 'req-9bcb29b2-e321-4479-a10e-bb0285ad3660'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.647929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2bad9bd51b247e78dbf42c6c6ab4528', 'request_id': 'req-9bcb29b2-e321-4479-a10e-bb0285ad3660'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.658060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] The token cache is being invalidate because project c2bad9bd51b247e78dbf42c6c6ab4528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:13:29.658345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.658983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:13:29.660500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1208/2414] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:13:29 2022] DELETE /identity/v3/projects/c2bad9bd51b247e78dbf42c6c6ab4528 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:13:39.313090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:39.655113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:43.371551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:43.373071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:43.429976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f6b9b-1442-49a8-8500-c51211d15453 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:43.481432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:43.482082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:43.482364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:43.482642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:43.484422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:43.556618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:43.558402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35410b9b-5c4c-4cec-8ca1-9efb3a7ec7ae service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:43.575097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1207/2415] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:46.893880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:46.895324 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:46.954557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb29b2-e321-4479-a10e-bb0285ad3660 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:46.958112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:46.958747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:46.959062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:46.959341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:46.962135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:47.020956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:47.022482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:47.032585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1209/2416] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:53.567859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=74 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:57.028507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=48 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:13:58.032991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:58.033440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:58.033539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:58.056247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] MFA Rules not processed for user `adb92379209e46e79de4df370183fa8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:13:58.057500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:58.065217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1208/2417] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:13:58.101660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:58.110802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a301-ebc7-48f0-855a-1ccdcd8d7f08 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:58.114383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:58.115050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:58.115339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:58.115620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:58.117384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:58.130399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:58.131857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:58.142633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1210/2418] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:13:58.964468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:13:58.965983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:58.978789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d5e6f-2302-471b-b5c6-5d9642250d08 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:13:58.982461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:13:58.983259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:13:58.983556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:13:58.983838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:13:58.985583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:13:59.055258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:13:59.057286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:13:59.067814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1209/2419] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:03.523524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:03.537877 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128795c-6a41-4184-aeb1-2f677535f7ad service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:03.545679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:03.548233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:03.557038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:03.558113 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:03.558629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:03.559336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:03.565679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:03.574590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8d498-3342-42ea-8c6b-bbe9810238de service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:03.588745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:03.590361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:03.592503 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:03.594369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:03.594664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:03.595140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] PATH_INFO: `/v3/users/6c904a413f6a4ba49686debac817c70d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:03.613593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1211/2420] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:03.647140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c904a413f6a4ba49686debac817c70d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:03.649022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:03.762458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c904a413f6a4ba49686debac817c70d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:03.763688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c904a413f6a4ba49686debac817c70d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:03.765872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c904a413f6a4ba49686debac817c70d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:03.792381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c904a413f6a4ba49686debac817c70d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:03.810182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d489c1fa-05ee-4491-ae67-284b86daed46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c904a413f6a4ba49686debac817c70d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:03.831771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1210/2421] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:03 2022] DELETE /identity/v3/users/6c904a413f6a4ba49686debac817c70d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:04.075285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.103998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da311c-a822-428c-a955-4dc160241fbd service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.181559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.182914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.183414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.184074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] PATH_INFO: `/v3/projects/ffaf09bc77914692bb965b55af538141` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.200360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffaf09bc77914692bb965b55af538141)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.202487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.261445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffaf09bc77914692bb965b55af538141', 'request_id': 'req-3c7b393e-956d-4c43-a7ff-c848ad2c4681'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:04.269751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffaf09bc77914692bb965b55af538141', 'request_id': 'req-3c7b393e-956d-4c43-a7ff-c848ad2c4681'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:04.280544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffaf09bc77914692bb965b55af538141', 'request_id': 'req-3c7b393e-956d-4c43-a7ff-c848ad2c4681'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:04.293836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] The token cache is being invalidate because project ffaf09bc77914692bb965b55af538141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:04.294125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:04.295828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-3c7b393e-956d-4c43-a7ff-c848ad2c4681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:04.297717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1212/2422] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:04 2022] DELETE /identity/v3/projects/ffaf09bc77914692bb965b55af538141 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:04.302449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.302815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.303365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.323711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:04.325151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:04.377709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:04.398186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1211/2423] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:04.406742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.407072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.407357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.425791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:04.427045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:04.436006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1213/2424] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:04.442198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.452875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1d766-6168-4e31-826c-a4f10966a638 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.458057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.458727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.459048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.459496 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.460230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:04.460545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.462313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.475607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1212/2425] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:04.480421 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.493497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69597b-750f-4cd5-b002-7232acca3193 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.500811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.501502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.501797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.502078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.502893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.504899 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.537821 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1214/2426] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:04 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:04.542475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.553542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94f51e-523b-4bc1-8d03-9b82eaec4755 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.624374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.625233 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.625724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.626177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.627260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.629908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.657629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1213/2427] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:04 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:04.661918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.676977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91fbb-2cb2-4f9b-91c8-d4193c7bf50d admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.683930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.685283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.685633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.685986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.686855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.688561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.702270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1215/2428] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:04.706149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.717214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7610920-65da-474d-908a-fda19092a5d1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.722087 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c72a20-47c2-4357-9856-3bf0b9d98da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.722753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c72a20-47c2-4357-9856-3bf0b9d98da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.723079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c72a20-47c2-4357-9856-3bf0b9d98da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.723365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c72a20-47c2-4357-9856-3bf0b9d98da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.724095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c72a20-47c2-4357-9856-3bf0b9d98da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.725703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c72a20-47c2-4357-9856-3bf0b9d98da3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.738942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1214/2429] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:04.743378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:04.755504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35ffe-7c5e-4908-8391-c8946b2c7863 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:04.761082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:04.762050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.762420 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.762919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] PATH_INFO: `/v3/projects/67b0ce27c73a4530a3b250590698e515/users/ff9ac774ade44e97bf942c25b953d9ce/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.775513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:04.778994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b0ce27c73a4530a3b250590698e515, user_id=ff9ac774ade44e97bf942c25b953d9ce, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:04.782055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:04.798507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1216/2430] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:04 2022] PUT /identity/v3/projects/67b0ce27c73a4530a3b250590698e515/users/ff9ac774ade44e97bf942c25b953d9ce/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:04.809764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:04.810065 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:04.810357 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:04.842415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] MFA Rules not processed for user `ff9ac774ade44e97bf942c25b953d9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:04.844154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:04.983466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1215/2431] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:04.996840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:05.001027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:05.111799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6640-5235-4433-b68c-280f552e3954 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:05.120650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb0c72-97d4-4ec0-bcfe-a16b51432e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:05.121442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb0c72-97d4-4ec0-bcfe-a16b51432e31 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:05.121734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb0c72-97d4-4ec0-bcfe-a16b51432e31 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:05.122014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb0c72-97d4-4ec0-bcfe-a16b51432e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:05.131792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb0c72-97d4-4ec0-bcfe-a16b51432e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:05.134169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb0c72-97d4-4ec0-bcfe-a16b51432e31 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:05.145439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1217/2432] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:05.365538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:05.415630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:05.435535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8605a-6b67-4c1a-b5c8-4842d78d0e37 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:05.440030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:05.440906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:05.441192 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:05.441471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] PATH_INFO: `/v3/users/ff9ac774ade44e97bf942c25b953d9ce` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:05.442712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9ac774ade44e97bf942c25b953d9ce)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:05.444829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:05.490161 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9ac774ade44e97bf942c25b953d9ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:05.490807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9ac774ade44e97bf942c25b953d9ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:05.491454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9ac774ade44e97bf942c25b953d9ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:05.498757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9ac774ade44e97bf942c25b953d9ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:05.510606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9ac774ade44e97bf942c25b953d9ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:05.523484 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1216/2433] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:05 2022] DELETE /identity/v3/users/ff9ac774ade44e97bf942c25b953d9ce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:05.528480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628431-4093-477f-9312-b6fe0120262d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:05.528803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628431-4093-477f-9312-b6fe0120262d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:05.529090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628431-4093-477f-9312-b6fe0120262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:05.547375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8d628431-4093-477f-9312-b6fe0120262d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:05.548573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8d628431-4093-477f-9312-b6fe0120262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:05.607350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1218/2434] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:05.617276 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:05.619104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:05.682358 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108f56-b4cf-43c8-b53b-bb9136e7add1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:05.686009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b3436-3fb5-41a7-a802-11a6a660e241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:05.686654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b3436-3fb5-41a7-a802-11a6a660e241 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:05.686969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b3436-3fb5-41a7-a802-11a6a660e241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:05.687252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b3436-3fb5-41a7-a802-11a6a660e241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:05.696262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b3436-3fb5-41a7-a802-11a6a660e241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:05.697691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b3436-3fb5-41a7-a802-11a6a660e241 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:05.707022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1217/2435] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:05.983281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628431-4093-477f-9312-b6fe0120262d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:05.984779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8d628431-4093-477f-9312-b6fe0120262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:05.997025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628431-4093-477f-9312-b6fe0120262d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.010646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.014910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.014910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.014910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] PATH_INFO: `/v3/projects/67b0ce27c73a4530a3b250590698e515` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.014910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b0ce27c73a4530a3b250590698e515)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.015944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.073254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b0ce27c73a4530a3b250590698e515', 'request_id': 'req-2304483d-c253-4996-96ab-d17dc518b72f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:06.080139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b0ce27c73a4530a3b250590698e515', 'request_id': 'req-2304483d-c253-4996-96ab-d17dc518b72f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:06.088778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b0ce27c73a4530a3b250590698e515', 'request_id': 'req-2304483d-c253-4996-96ab-d17dc518b72f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:06.096263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:06.100565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] The token cache is being invalidate because project 67b0ce27c73a4530a3b250590698e515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:06.101304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:06.104642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-2304483d-c253-4996-96ab-d17dc518b72f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:06.108390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1219/2436] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:05 2022] DELETE /identity/v3/projects/67b0ce27c73a4530a3b250590698e515 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:06.111643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.112255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.112255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.130444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:06.131810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:06.212198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1218/2437] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:06.219793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.220095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.220390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.238730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:06.240002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:06.248864 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1220/2438] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:06.253160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.263949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36266d3-72f6-4aa8-b6ef-04109cb558c3 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.268747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.269547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.269906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.270280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.271129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:06.271511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.273251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.275100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:06.287359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1219/2439] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:06.292091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.301408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06c6af-09e0-4879-811c-6b23e1247eae None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.307044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.307807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.308159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.308508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.309271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.311314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.338016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1221/2440] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:06.341949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.351996 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46baf4f-0cca-4617-a410-6fa0192cba15 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.411685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.412305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.412656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.412925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.413670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.415683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.437989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1220/2441] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:06.442196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.453544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3da42-04f6-4530-b11a-6bf3c97403a9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.458318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.459103 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.459398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.459674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.460553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.462387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.473517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1222/2442] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:06.477486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.488029 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f85be4-984b-459a-9cf7-985a79239758 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.494720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f0f55-8dc1-4b27-93ac-102df577323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.495439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f0f55-8dc1-4b27-93ac-102df577323f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.495728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f0f55-8dc1-4b27-93ac-102df577323f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.496009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f0f55-8dc1-4b27-93ac-102df577323f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.496770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f0f55-8dc1-4b27-93ac-102df577323f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.498389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f0f55-8dc1-4b27-93ac-102df577323f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.509568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1221/2443] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:06.514275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.527519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d2e14-c6f5-4179-a4ad-a2a577c84d06 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.532335 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.533550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.533930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.534312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] PATH_INFO: `/v3/projects/403198949d924a11ab5896011383c63a/users/ed0d44994f924710881e90ec64d36924/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.556601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=403198949d924a11ab5896011383c63a, user_id=ed0d44994f924710881e90ec64d36924, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.559601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.573651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1223/2444] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:06 2022] PUT /identity/v3/projects/403198949d924a11ab5896011383c63a/users/ed0d44994f924710881e90ec64d36924/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:06.586008 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.586314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.587114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.618774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] MFA Rules not processed for user `ed0d44994f924710881e90ec64d36924`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:06.620046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:06.691629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1222/2445] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:06.696053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.762328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c85bd-d860-4688-aa41-71392b56d94a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.767151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.767820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.768107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.768392 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.769098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.771051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.797945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1224/2446] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:06.801908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.813456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74c1e3-4f78-45c8-a215-935b1e46a415 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.875352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.876052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.876342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.876662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.877339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.879331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.887349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:06.900787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1223/2447] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:06 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:06.904535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.915651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ed428-0c40-4bfe-a698-2fa1cf3ed78a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.920318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.920975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.921267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.921688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.922438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.924108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.934798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1225/2448] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:06.938832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:06.949362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eac5b0-a715-46b0-9ce9-08256d356ded admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:06.953925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:06.954803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:06.955130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:06.955409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] PATH_INFO: `/v3/projects/a491c644b1854bb6937078991c4a1324/users/8d38472e42ca4d43a5b3f7c63372bc4d/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:06.970688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a491c644b1854bb6937078991c4a1324, user_id=8d38472e42ca4d43a5b3f7c63372bc4d, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:06.973384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:06.986428 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1224/2449] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:06 2022] PUT /identity/v3/projects/a491c644b1854bb6937078991c4a1324/users/8d38472e42ca4d43a5b3f7c63372bc4d/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:06.990371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:07.000440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e03ba-122b-4e31-bb2b-cbd8a18a6008 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:07.058048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01924b0b-b3b1-4d65-a83d-017e640e7dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:07.058538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01924b0b-b3b1-4d65-a83d-017e640e7dce admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:07.058944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01924b0b-b3b1-4d65-a83d-017e640e7dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:07.059143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01924b0b-b3b1-4d65-a83d-017e640e7dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:07.060809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01924b0b-b3b1-4d65-a83d-017e640e7dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:07.062558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01924b0b-b3b1-4d65-a83d-017e640e7dce admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:07.073742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1226/2450] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:07.077884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:07.088793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ac10c-500b-4fdf-beac-b77a07a43265 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:07.093974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:07.094873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:07.095165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:07.095446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] PATH_INFO: `/v3/projects/a491c644b1854bb6937078991c4a1324/users/8d38472e42ca4d43a5b3f7c63372bc4d/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:07.097417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a491c644b1854bb6937078991c4a1324, user_id=8d38472e42ca4d43a5b3f7c63372bc4d, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:07.100126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:07.110557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1225/2451] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:07 2022] PUT /identity/v3/projects/a491c644b1854bb6937078991c4a1324/users/8d38472e42ca4d43a5b3f7c63372bc4d/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:07.121228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:07.121546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:07.121829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:07.134813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:07.149380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] MFA Rules not processed for user `8d38472e42ca4d43a5b3f7c63372bc4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:07.150601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:07.218256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1227/2452] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:07.230000 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:07.232381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:07.306104 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc1268-8ef4-4c3d-be84-45d4622cee2d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:07.315257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:07.316091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:07.316383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:07.316668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:07.326380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:07.327836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:07.338266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1226/2453] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:07.421793 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:07.423332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:07.438142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ae7b6-5d21-4f61-a2a5-fa692a430b4c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:07.444514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:07.445173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:07.445463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:07.445744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:07.527870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:07.529516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:07.538395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1228/2454] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:08.609740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:08.611185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:08.670352 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9ebee-ac5a-4a3c-9410-50dbf87eea58 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:08.674033 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:08.674680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:08.675351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:08.675351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:08.677467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:08.745536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:08.747009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:08.756668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1227/2455] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:12.021328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.022744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:12.038082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75465537-4a6b-47ef-9327-253c723034c7 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.043322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.044207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.044620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.044912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] PATH_INFO: `/v3/users/6aebb057a324492699141a878793c097` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.061048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aebb057a324492699141a878793c097)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.063023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.115670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aebb057a324492699141a878793c097'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.116325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aebb057a324492699141a878793c097'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.125078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aebb057a324492699141a878793c097'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.125770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aebb057a324492699141a878793c097'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.134673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d75d2a93-cc25-4b38-9217-b91c69a8a919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aebb057a324492699141a878793c097'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.144759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:12.161258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1229/2456] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:12 2022] DELETE /identity/v3/users/6aebb057a324492699141a878793c097 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:12.358139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.368948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444bb02-c111-4709-a6a7-e7b263eb62a1 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.372995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:12.429965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.430979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.431272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.431555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] PATH_INFO: `/v3/projects/b7d38f1b5a8a43ea8677f3ce053347da` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.441814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d38f1b5a8a43ea8677f3ce053347da)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.443800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.493991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d38f1b5a8a43ea8677f3ce053347da', 'request_id': 'req-aa75bad8-c65b-4012-909d-049ecdbb1f68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.499837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d38f1b5a8a43ea8677f3ce053347da', 'request_id': 'req-aa75bad8-c65b-4012-909d-049ecdbb1f68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.513969 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d38f1b5a8a43ea8677f3ce053347da', 'request_id': 'req-aa75bad8-c65b-4012-909d-049ecdbb1f68'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.527223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] The token cache is being invalidate because project b7d38f1b5a8a43ea8677f3ce053347da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:12.527508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.528121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-aa75bad8-c65b-4012-909d-049ecdbb1f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:12.529656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1228/2457] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:12 2022] DELETE /identity/v3/projects/b7d38f1b5a8a43ea8677f3ce053347da => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:12.534620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.534957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.535269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.553919 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:12.555230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:12.615745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1230/2458] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:12.623662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.623959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.624246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.642512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:12.643767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:12.652605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1229/2459] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:12.656641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.666439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f537-29af-4fa9-9ccb-f9bd78c3b1a8 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.671152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.671810 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.672101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.672389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.673114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:12.673432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.675094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.686613 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1231/2460] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:12.690889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.700449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9a0b5-bfa9-4098-998c-6e51b0699902 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.704994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.705627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.705912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.706190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.706908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.708830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.735023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1230/2461] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:12 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:12.739246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.749424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8db7a5-cf2a-4cba-85de-d45430bf13d9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.806008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.806678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.807087 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.807373 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.808084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.810041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.830523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1232/2462] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:12 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:12.834654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.855334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb96a97-feef-4304-86b1-4f41bd9f4cf3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.860369 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.861030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.861325 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.861607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.862332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.863998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.879440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1231/2463] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:12.885245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.901298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211961-a463-47b5-9aca-3698779720f0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.906162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db90bd-4987-4173-9802-f04f9aff36de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.906854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db90bd-4987-4173-9802-f04f9aff36de admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.907160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db90bd-4987-4173-9802-f04f9aff36de admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.907593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db90bd-4987-4173-9802-f04f9aff36de admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.908164 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db90bd-4987-4173-9802-f04f9aff36de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.909814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db90bd-4987-4173-9802-f04f9aff36de admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.921696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1233/2464] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:12.926020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:12.938032 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4b77-375d-41e7-a6e3-713206ebbb08 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:12.942661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:12.943599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.943888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.944174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] PATH_INFO: `/v3/projects/b741f6cd02d24da2ab07ebb511fcb381/users/94c4597335154001bcc0bfd2bae15c95/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:12.959676 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b741f6cd02d24da2ab07ebb511fcb381, user_id=94c4597335154001bcc0bfd2bae15c95, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:12.962850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:12.976912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1232/2465] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:12 2022] PUT /identity/v3/projects/b741f6cd02d24da2ab07ebb511fcb381/users/94c4597335154001bcc0bfd2bae15c95/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:12.988275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:12.988663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:12.988956 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:13.017710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] MFA Rules not processed for user `94c4597335154001bcc0bfd2bae15c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:13.018734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:13.063073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:13.084258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1234/2466] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:13.095625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:13.097349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:13.126867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:13.173732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeadcce-5814-4a1b-826b-cdf92212be71 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:13.179264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602d3b-6b60-4948-8168-a17baa586438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:13.179908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602d3b-6b60-4948-8168-a17baa586438 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:13.180734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602d3b-6b60-4948-8168-a17baa586438 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:13.181013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602d3b-6b60-4948-8168-a17baa586438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:13.188583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602d3b-6b60-4948-8168-a17baa586438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:13.190166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602d3b-6b60-4948-8168-a17baa586438 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:13.208200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1233/2467] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:23.081407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:23.195085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=23 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:24.456326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:24.457755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:24.521677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8f8fb-0c9d-4d7d-b50b-268723b21de2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:24.526261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:24.527567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:24.527855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:24.528134 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] PATH_INFO: `/v3/users/ed0d44994f924710881e90ec64d36924` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:24.529353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0d44994f924710881e90ec64d36924)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:24.531342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:24.578597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0d44994f924710881e90ec64d36924'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.579258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0d44994f924710881e90ec64d36924'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.585954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0d44994f924710881e90ec64d36924'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.586581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0d44994f924710881e90ec64d36924'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.594812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0d44994f924710881e90ec64d36924'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.614158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1235/2468] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:24 2022] DELETE /identity/v3/users/ed0d44994f924710881e90ec64d36924 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:24.618812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:24.619306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:24.619406 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:24.641718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:24.644215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:24.708022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1234/2469] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:24.717572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:24.719253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:24.732972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:24.733076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:24.747124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593aedd-6728-4687-8da9-08d48e3fffb7 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:24.753489 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:24.754619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:24.754944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:24.755229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] PATH_INFO: `/v3/users/94c4597335154001bcc0bfd2bae15c95` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:24.756460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c4597335154001bcc0bfd2bae15c95)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:24.758407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:24.783333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dd2cc-42f9-4490-ad8f-dbea3d624af0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:24.787484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9c22d-a526-4b07-887b-bf72e10ea483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:24.788223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9c22d-a526-4b07-887b-bf72e10ea483 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:24.788593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9c22d-a526-4b07-887b-bf72e10ea483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:24.788880 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9c22d-a526-4b07-887b-bf72e10ea483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:24.799795 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9c22d-a526-4b07-887b-bf72e10ea483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:24.801235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9c22d-a526-4b07-887b-bf72e10ea483 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:24.809274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c4597335154001bcc0bfd2bae15c95'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.810015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c4597335154001bcc0bfd2bae15c95'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.810694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c4597335154001bcc0bfd2bae15c95'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.811524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1236/2470] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:24.817981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c4597335154001bcc0bfd2bae15c95'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.830882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c4597335154001bcc0bfd2bae15c95'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:24.845384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1235/2471] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:24 2022] DELETE /identity/v3/users/94c4597335154001bcc0bfd2bae15c95 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:24.850318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:24.850680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:24.851071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:24.869904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:24.871693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:24.890963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:24.937847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1237/2472] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:24.947956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:24.950431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:25.041534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb617f-7009-4e6b-a3b1-53fa4625dc0a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.046305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.047647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.047937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.048218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.059573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.061028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.071027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1236/2473] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:25.274774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.276446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:25.293656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abd60-b5ff-4e21-b8ec-5b80f6e91096 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.299384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.300264 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.300555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.300838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] PATH_INFO: `/v3/projects/403198949d924a11ab5896011383c63a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.302092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=403198949d924a11ab5896011383c63a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.304004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.378900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403198949d924a11ab5896011383c63a', 'request_id': 'req-d980c0e7-ea85-40e9-878c-ed937debad90'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.386008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403198949d924a11ab5896011383c63a', 'request_id': 'req-d980c0e7-ea85-40e9-878c-ed937debad90'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.397012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403198949d924a11ab5896011383c63a', 'request_id': 'req-d980c0e7-ea85-40e9-878c-ed937debad90'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.411378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] The token cache is being invalidate because project 403198949d924a11ab5896011383c63a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:25.411690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.412345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.413891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1238/2474] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:25 2022] DELETE /identity/v3/projects/403198949d924a11ab5896011383c63a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:25.418604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.429503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d51d-b1f5-4e8f-9395-21a7caaebc5d service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.442361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:25.492024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.492808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.493091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.493367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] PATH_INFO: `/v3/users/8d38472e42ca4d43a5b3f7c63372bc4d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.494553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d38472e42ca4d43a5b3f7c63372bc4d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.497356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.516266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.517740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:25.533295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980c0e7-ea85-40e9-878c-ed937debad90 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.539491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.541014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.541601 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.541741 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] PATH_INFO: `/v3/projects/b741f6cd02d24da2ab07ebb511fcb381` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.543522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b741f6cd02d24da2ab07ebb511fcb381)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.545524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.554077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d38472e42ca4d43a5b3f7c63372bc4d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.554791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d38472e42ca4d43a5b3f7c63372bc4d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.555480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d38472e42ca4d43a5b3f7c63372bc4d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.563437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d38472e42ca4d43a5b3f7c63372bc4d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.575139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-afae840d-ef11-4191-b238-04b5e85f169e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d38472e42ca4d43a5b3f7c63372bc4d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.590274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1237/2475] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:25 2022] DELETE /identity/v3/users/8d38472e42ca4d43a5b3f7c63372bc4d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:25.602826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b741f6cd02d24da2ab07ebb511fcb381', 'request_id': 'req-902a02e5-be8f-4b80-aa42-1ce60025532d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.613121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b741f6cd02d24da2ab07ebb511fcb381', 'request_id': 'req-902a02e5-be8f-4b80-aa42-1ce60025532d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.623865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b741f6cd02d24da2ab07ebb511fcb381', 'request_id': 'req-902a02e5-be8f-4b80-aa42-1ce60025532d'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.636148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] The token cache is being invalidate because project b741f6cd02d24da2ab07ebb511fcb381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:25.636636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.637385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-902a02e5-be8f-4b80-aa42-1ce60025532d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:25.639092 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1239/2476] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:25 2022] DELETE /identity/v3/projects/b741f6cd02d24da2ab07ebb511fcb381 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:25.644089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.644812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.645226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.664332 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:25.665653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:25.727992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1238/2477] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:25.737924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.738214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.738526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.758715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:25.760024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:25.768569 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1240/2478] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:25.772626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.782625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a9cbf-9141-410d-b913-bd5c15265a04 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.787668 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.788324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.788609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.788889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.789608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:25.789917 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.791559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.804521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1239/2479] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:25.808747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.818266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bab23-26d9-42f3-87de-db73f013b3ba None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.823015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.823769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.824130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.824495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.825249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.827309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.849413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:25.858442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1241/2480] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:25 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:25.862649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.875255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.876426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b56d9-a8ff-4c1f-82cb-a7f915fd7b26 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.877411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:25.901378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:25.945488 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.946190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.946481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.946763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.947509 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.948028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584500d-9ebb-4808-a300-9d80185049bd admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.950190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.953311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.954172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.954454 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.954733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] PATH_INFO: `/v3/projects/a491c644b1854bb6937078991c4a1324` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.955936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a491c644b1854bb6937078991c4a1324)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.957866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:25.974621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1240/2481] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:25 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:25.975636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:25.987729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaae2e-d3ea-4fe1-9ebe-c12292ec2654 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:25.992749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:25.993410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:25.993700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:25.994030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:25.994703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:25.996625 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.006194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a491c644b1854bb6937078991c4a1324', 'request_id': 'req-91cc8acb-8115-4c11-8461-70d26ba4ae1e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:26.013136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a491c644b1854bb6937078991c4a1324', 'request_id': 'req-91cc8acb-8115-4c11-8461-70d26ba4ae1e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:26.020135 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1241/2482] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:26.021412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a491c644b1854bb6937078991c4a1324', 'request_id': 'req-91cc8acb-8115-4c11-8461-70d26ba4ae1e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:26.024102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.033572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] The token cache is being invalidate because project a491c644b1854bb6937078991c4a1324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:26.034055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:26.034833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-91cc8acb-8115-4c11-8461-70d26ba4ae1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:26.035253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa422b-f6bd-487a-b61d-f1ae7370779c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.036660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1242/2483] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:25 2022] DELETE /identity/v3/projects/a491c644b1854bb6937078991c4a1324 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:26.040965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae007-ea4d-40e5-961f-e670136da126 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.041280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae007-ea4d-40e5-961f-e670136da126 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.041567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae007-ea4d-40e5-961f-e670136da126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.059833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-803ae007-ea4d-40e5-961f-e670136da126 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:26.061031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-803ae007-ea4d-40e5-961f-e670136da126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.098317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.099017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.099308 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.099585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.100303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.101911 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.112137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1242/2484] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:14:26.116113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.117807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.123817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1243/2485] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:26.130023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807091-0eb7-47d1-b824-98fa031a60e6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.132610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.132910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.133219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.135023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1db74a-31e1-404d-9ca4-aa8c0c4ffdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.135923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1db74a-31e1-404d-9ca4-aa8c0c4ffdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.136742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1db74a-31e1-404d-9ca4-aa8c0c4ffdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.137024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1db74a-31e1-404d-9ca4-aa8c0c4ffdd8 admin admin] PATH_INFO: `/v3/projects/52cb240a8a0f4174b55b2f31c8984601/users/f0c93a9b14af4e40ac2d93ee31e1697e/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.152457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:26.152871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1db74a-31e1-404d-9ca4-aa8c0c4ffdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52cb240a8a0f4174b55b2f31c8984601, user_id=f0c93a9b14af4e40ac2d93ee31e1697e, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.153875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.155692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1db74a-31e1-404d-9ca4-aa8c0c4ffdd8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.162192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1244/2486] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:26.166385 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.169120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1243/2487] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:26 2022] PUT /identity/v3/projects/52cb240a8a0f4174b55b2f31c8984601/users/f0c93a9b14af4e40ac2d93ee31e1697e/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:26.175915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abbae9-fdd8-48bf-a231-6466afe4b031 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.179061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.179351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.179639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.197052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:26.206900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] MFA Rules not processed for user `f0c93a9b14af4e40ac2d93ee31e1697e`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:26.208153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.238112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.238785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.239111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.239402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.240131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:26.240445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.242123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.258671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1245/2488] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:14:26.263667 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.274272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7a415-b25e-4717-bb0d-c9ae06229a1a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.279733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.280449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.280743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.281031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.281723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.283688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.293317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1244/2489] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:26.294555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.300595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:26.309251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b600e0-e35f-4e0c-afdc-0bf546753353 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.315816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.317170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.317419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.317693 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.318401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.319298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1246/2490] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:26.320575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.323377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.354549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1245/2491] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:26.356411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.399996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9f224-901a-4e84-a717-f2a0b29a1c04 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.427998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f70b5-a8d7-467c-8a17-f8f06e90fa56 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.434252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.434725 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.435074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.435360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.436052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.438005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.462962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1246/2492] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:26.467299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.474590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.475929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.476225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.476933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.477771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.480208 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843a8b0-d012-4f76-b5a4-a45851f1ba1a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.480357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.484025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.484694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.484983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.485324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.486139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.488364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.502700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1247/2493] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:26.506554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1247/2494] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:26.507164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.510323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.517924 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25340a8c-2804-4d5d-96c6-5bcab33d1e93 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.522055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f933-ec70-43a3-ad09-723a54e1dc6f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.522665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.523456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.523741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.524024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] PATH_INFO: `/v3/projects/0b0b1d61310b4ad5831add4f053d3041/users/26c490a3010940b99146e1878c183589/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.528273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.529063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.529448 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.529830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.530655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.532475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.540387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0b1d61310b4ad5831add4f053d3041, user_id=26c490a3010940b99146e1878c183589, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.543884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.546890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1248/2495] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:14:26.548553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.560467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eecd92-48cb-43aa-956e-950bb10468c8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.568165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1248/2496] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:26 2022] PUT /identity/v3/projects/0b0b1d61310b4ad5831add4f053d3041/users/26c490a3010940b99146e1878c183589/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:26.571651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.582742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:26.641852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.642740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.643062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.643348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.644089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.645727 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.650683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56eef9-021c-4d5f-94c2-31fc2cc5004e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.658011 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1249/2497] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:26.660123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.661463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.661896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.662143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.662744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.664138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.665997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.677403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7432-bc25-4bf4-a30a-d3756fae16eb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.678879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1249/2498] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:14:26.679568 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.681293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.686500 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.687555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.687839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.688118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] PATH_INFO: `/v3/projects/322086f968e543edb96fe5792cec6223/users/fb3b5ab7befd4894a05375c463645040/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.703698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] RBAC: Authorizing `identity:create_grant(project_id=322086f968e543edb96fe5792cec6223, user_id=fb3b5ab7befd4894a05375c463645040, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.706395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.717230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:26.720311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1250/2499] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:26 2022] PUT /identity/v3/projects/322086f968e543edb96fe5792cec6223/users/fb3b5ab7befd4894a05375c463645040/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:26.727717 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.744340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760473-c155-4cb4-94b3-3770ef6010c2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.792102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08855424-9bde-4e4c-aea0-0324a98b458c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:26.794214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.794899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.795212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.795475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.797287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.797566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65b50c-4c96-4db1-a212-719a449d8387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:26.798445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65b50c-4c96-4db1-a212-719a449d8387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.798730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65b50c-4c96-4db1-a212-719a449d8387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.799035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65b50c-4c96-4db1-a212-719a449d8387 admin admin] PATH_INFO: `/v3/projects/0b0b1d61310b4ad5831add4f053d3041/users/26c490a3010940b99146e1878c183589/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.801075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65b50c-4c96-4db1-a212-719a449d8387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0b1d61310b4ad5831add4f053d3041, user_id=26c490a3010940b99146e1878c183589, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.803765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65b50c-4c96-4db1-a212-719a449d8387 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.815227 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1251/2500] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:26 2022] PUT /identity/v3/projects/0b0b1d61310b4ad5831add4f053d3041/users/26c490a3010940b99146e1878c183589/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:26.821359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.821359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.821359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.843912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] MFA Rules not processed for user `fb3b5ab7befd4894a05375c463645040`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:26.845137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.876966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:26.878521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93049ea0-d56e-40b6-a932-6186c7ef9943 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:26.888882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1250/2501] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:26.890176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:26.890467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:26.890751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:26.920726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] MFA Rules not processed for user `26c490a3010940b99146e1878c183589`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:26.922411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:26.922827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1252/2502] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:26.927471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:26.951778 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:27.021043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1251/2503] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:27.033409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.034515 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620880-7cd3-4384-a0a5-ca15fa79e7bf None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.035061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:27.040118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.041091 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.041441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.042892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.043597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.046177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.066822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:27.090799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1253/2504] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:26 2022] POST /identity/v3/projects => generated 393 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:27.091830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.108745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f420-b5df-449e-b12f-a88f7549c69b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.165003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6367ae2-9c1c-4fc9-b8ba-54353fff3a0c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.170322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.171007 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.171294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.171913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.172286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.173537 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.173914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.174293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.175128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.177207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.203377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1254/2505] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:27 2022] POST /identity/v3/users => generated 303 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:27.204622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.216993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7efef0-e30b-44e0-b209-8305acbc7db4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.222074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.222794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.223119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.223399 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.224151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.225753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.241088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1255/2506] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:27.242094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.252508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.254199 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.258914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dc17d-bf4c-42ee-9966-34ab5297f0d4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.267036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.268151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1252/2507] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:27.270054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.270351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.270639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] PATH_INFO: `/v3/projects/6368b89f87a641a7b11112e12ddea08e/users/bfe9c15bc1864ed0954701fa7c4d316c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.299274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6368b89f87a641a7b11112e12ddea08e, user_id=bfe9c15bc1864ed0954701fa7c4d316c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.303181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.318283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1256/2508] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:27 2022] PUT /identity/v3/projects/6368b89f87a641a7b11112e12ddea08e/users/bfe9c15bc1864ed0954701fa7c4d316c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:27.322865 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.336075 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086e22c-2c5b-4e68-bd2b-aa1ffa5491fe service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.350958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.402581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.403311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.403605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.403884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.404772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.407751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.424828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e78ab2-d28d-4eac-a62b-20a18c5c5dd5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.425112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1253/2509] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:14:27.428855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.429340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.429499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.429783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.430176 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.440171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc5ef-807e-4603-93ac-065d002b6727 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.441382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:27.445812 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7141abd-29ae-499f-a60c-af0e04d800c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.446810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7141abd-29ae-499f-a60c-af0e04d800c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.447377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7141abd-29ae-499f-a60c-af0e04d800c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.447777 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7141abd-29ae-499f-a60c-af0e04d800c9 admin admin] PATH_INFO: `/v3/projects/6368b89f87a641a7b11112e12ddea08e/users/bfe9c15bc1864ed0954701fa7c4d316c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.451346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7141abd-29ae-499f-a60c-af0e04d800c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6368b89f87a641a7b11112e12ddea08e, user_id=bfe9c15bc1864ed0954701fa7c4d316c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.455386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7141abd-29ae-499f-a60c-af0e04d800c9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.469093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1254/2510] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:27 2022] PUT /identity/v3/projects/6368b89f87a641a7b11112e12ddea08e/users/bfe9c15bc1864ed0954701fa7c4d316c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:27.479370 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.479662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.479981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.502601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:27.506367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.507905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.512546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] MFA Rules not processed for user `bfe9c15bc1864ed0954701fa7c4d316c`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:27.514294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:27.518559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1257/2511] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:14:27.598365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1255/2512] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:27.610433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.611988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:27.625662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df6e63-44d5-4cc2-9b18-639c2e048f71 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.693098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.693927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.694283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.694721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.705002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.706501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.717625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1258/2513] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:27.770343 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:27.780463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1dc31-95c9-43ae-9b0a-754a1dd45ca8 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:27.784396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62adf17-8504-4a96-9fa4-f74de4afd9f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:27.785119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62adf17-8504-4a96-9fa4-f74de4afd9f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:27.785478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62adf17-8504-4a96-9fa4-f74de4afd9f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:27.785829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62adf17-8504-4a96-9fa4-f74de4afd9f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:27.845103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62adf17-8504-4a96-9fa4-f74de4afd9f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:27.846541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62adf17-8504-4a96-9fa4-f74de4afd9f9 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:27.856407 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1256/2514] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:31.290364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:31.351710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b6a37-2472-4a13-9477-396567995783 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:31.356911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:31.357776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:31.358064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:31.358350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] PATH_INFO: `/v3/users/fb3b5ab7befd4894a05375c463645040` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:31.359563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3b5ab7befd4894a05375c463645040)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:31.361493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:31.407710 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3b5ab7befd4894a05375c463645040'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.408275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3b5ab7befd4894a05375c463645040'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.415225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3b5ab7befd4894a05375c463645040'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.415840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3b5ab7befd4894a05375c463645040'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.417603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:31.424953 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3b5ab7befd4894a05375c463645040'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.442453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1259/2515] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:31 2022] DELETE /identity/v3/users/fb3b5ab7befd4894a05375c463645040 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:31.446943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:31.447418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:31.447682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:31.467895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:31.469379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:31.529593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1257/2516] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:31.539255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:31.540714 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:31.598789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469c063-e4c1-4287-ab97-a1cf97ae6739 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:31.602497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:31.603220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:31.603597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:31.603848 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:31.614603 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:31.616121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:31.625644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1260/2517] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:31.886151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:31.887638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:31.899446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1d51f-7d92-4015-964b-feb9aad8898d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:31.904549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:31.905417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:31.905698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:31.905975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] PATH_INFO: `/v3/projects/322086f968e543edb96fe5792cec6223` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:31.907377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] RBAC: Authorizing `identity:delete_project(project_id=322086f968e543edb96fe5792cec6223)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:31.909314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:31.944520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:31.959099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322086f968e543edb96fe5792cec6223', 'request_id': 'req-878ca3c9-c469-44f9-a81f-4a18b929843e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.965871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322086f968e543edb96fe5792cec6223', 'request_id': 'req-878ca3c9-c469-44f9-a81f-4a18b929843e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.972313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322086f968e543edb96fe5792cec6223', 'request_id': 'req-878ca3c9-c469-44f9-a81f-4a18b929843e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.984322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] The token cache is being invalidate because project 322086f968e543edb96fe5792cec6223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:31.984621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.985251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:31.986793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1258/2518] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:31 2022] DELETE /identity/v3/projects/322086f968e543edb96fe5792cec6223 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:31.990809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.000954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ab76b-9132-45d0-9c6b-bed4e102c415 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.061394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.062271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.062558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.062957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] PATH_INFO: `/v3/users/bfe9c15bc1864ed0954701fa7c4d316c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.064154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfe9c15bc1864ed0954701fa7c4d316c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.066110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.112825 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfe9c15bc1864ed0954701fa7c4d316c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.113495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfe9c15bc1864ed0954701fa7c4d316c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.120376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfe9c15bc1864ed0954701fa7c4d316c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.121007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfe9c15bc1864ed0954701fa7c4d316c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.129192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8820d80e-3f7d-4b99-9fb3-af44712f9eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfe9c15bc1864ed0954701fa7c4d316c'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.140466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:32.148702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1261/2519] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:31 2022] DELETE /identity/v3/users/bfe9c15bc1864ed0954701fa7c4d316c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:32.420448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.421993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:32.437081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca3c9-c469-44f9-a81f-4a18b929843e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.494245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.495145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.496757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.496757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] PATH_INFO: `/v3/projects/6368b89f87a641a7b11112e12ddea08e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.497627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6368b89f87a641a7b11112e12ddea08e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.499598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.546570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6368b89f87a641a7b11112e12ddea08e', 'request_id': 'req-5d033f63-fe8f-43e4-8b64-6076e82ce94e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.552174 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6368b89f87a641a7b11112e12ddea08e', 'request_id': 'req-5d033f63-fe8f-43e4-8b64-6076e82ce94e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.558587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6368b89f87a641a7b11112e12ddea08e', 'request_id': 'req-5d033f63-fe8f-43e4-8b64-6076e82ce94e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.570470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] The token cache is being invalidate because project 6368b89f87a641a7b11112e12ddea08e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:32.570760 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.571411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-5d033f63-fe8f-43e4-8b64-6076e82ce94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:32.572941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1259/2520] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:32 2022] DELETE /identity/v3/projects/6368b89f87a641a7b11112e12ddea08e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:32.577905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.578376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.578494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.597254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:32.598462 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:32.657898 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1262/2521] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:32.666745 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.667071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.667990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.686372 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:32.687619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:32.695531 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1260/2522] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:32.700128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.709734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988464f-e3fd-468c-92f2-b834348bdfc3 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.714995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.715697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.715986 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.716266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.716985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:32.717297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.719183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.730686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1263/2523] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:32.734803 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.744046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf2742-f919-4ea2-b1f0-f30e98d75f77 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.749142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.749728 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.750020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.750305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.751036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.752952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.779468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1261/2524] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:32 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:32.783752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.794249 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed738a05-488f-4758-ba85-b71fe0e5363b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.857300 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.857975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.858266 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.858549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.859278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.861295 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.883405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1264/2525] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:32 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:32.887840 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.897918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a04560-1466-4957-9631-0eeeb10701a6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.902738 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.903417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.903697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.903981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.904697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.906371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.916723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1262/2526] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:32.921476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.931615 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d1e84-174d-4689-bd87-88ef531f3dd5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.936829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6f84-1d16-408a-be33-0211a9317df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.937484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6f84-1d16-408a-be33-0211a9317df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.937774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6f84-1d16-408a-be33-0211a9317df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.938055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6f84-1d16-408a-be33-0211a9317df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.938781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc6f84-1d16-408a-be33-0211a9317df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.940415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc6f84-1d16-408a-be33-0211a9317df0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:32.950470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1265/2527] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:32.954537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:32.964595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae694ad-bc13-481e-92c4-0b6be2893777 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:32.969597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:32.970465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:32.970753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:32.971069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] PATH_INFO: `/v3/projects/2efce33548964faaa1e153dc11861622/users/ce80d151b3fb4163b19fec7365e5c67a/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:32.974121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:32.987265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2efce33548964faaa1e153dc11861622, user_id=ce80d151b3fb4163b19fec7365e5c67a, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:32.989919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:33.003264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1263/2528] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:32 2022] PUT /identity/v3/projects/2efce33548964faaa1e153dc11861622/users/ce80d151b3fb4163b19fec7365e5c67a/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:33.013639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:33.013934 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:33.014214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:33.041045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] MFA Rules not processed for user `ce80d151b3fb4163b19fec7365e5c67a`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:33.042743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:33.092683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:33.111501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1266/2529] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:33.122214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:33.123960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:33.179623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba59bfe-43cb-4413-9444-a633723ce842 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:33.183416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23408ec-a6d7-43e8-9e5d-0d969e330697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:33.184057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23408ec-a6d7-43e8-9e5d-0d969e330697 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:33.184342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23408ec-a6d7-43e8-9e5d-0d969e330697 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:33.184622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23408ec-a6d7-43e8-9e5d-0d969e330697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:33.192843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23408ec-a6d7-43e8-9e5d-0d969e330697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:33.194280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23408ec-a6d7-43e8-9e5d-0d969e330697 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:33.203222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1264/2530] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:14:43.108493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:43.199417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:45.749173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:45.811879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4c28d-2175-419f-9e1b-21236ffa5b86 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:45.816398 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:45.817265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:45.817551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:45.817831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] PATH_INFO: `/v3/users/ce80d151b3fb4163b19fec7365e5c67a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:45.819085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce80d151b3fb4163b19fec7365e5c67a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:45.821024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:45.867495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce80d151b3fb4163b19fec7365e5c67a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:45.868344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce80d151b3fb4163b19fec7365e5c67a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:45.876545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce80d151b3fb4163b19fec7365e5c67a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:45.877180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce80d151b3fb4163b19fec7365e5c67a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:45.883677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce80d151b3fb4163b19fec7365e5c67a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:45.900495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1267/2531] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:45 2022] DELETE /identity/v3/users/ce80d151b3fb4163b19fec7365e5c67a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:45.905508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:45.906027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:45.906128 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:45.923460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:45.924666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:45.983848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1265/2532] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:45.993357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:45.994750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:46.055202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6c963-8265-492f-a0da-007b9f8d1e35 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.058878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b80e-0556-460d-b62c-bd54760f505e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.059525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b80e-0556-460d-b62c-bd54760f505e service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.059808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b80e-0556-460d-b62c-bd54760f505e service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.060083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b80e-0556-460d-b62c-bd54760f505e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.069021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b80e-0556-460d-b62c-bd54760f505e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.070466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b80e-0556-460d-b62c-bd54760f505e service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.079828 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1268/2533] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:46.350825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.352462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:46.376477 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba85c4a-7c6e-44cd-970b-6b375602252f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.381389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.382259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.382548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.382830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] PATH_INFO: `/v3/projects/2efce33548964faaa1e153dc11861622` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.384044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2efce33548964faaa1e153dc11861622)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.386549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.434190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2efce33548964faaa1e153dc11861622', 'request_id': 'req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:46.439644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2efce33548964faaa1e153dc11861622', 'request_id': 'req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:46.445628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2efce33548964faaa1e153dc11861622', 'request_id': 'req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:46.459398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] The token cache is being invalidate because project 2efce33548964faaa1e153dc11861622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:46.459688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:46.460340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-6cbfa7e0-17f0-4515-a1a5-0a3a6ca72e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:46.461879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1266/2534] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:46 2022] DELETE /identity/v3/projects/2efce33548964faaa1e153dc11861622 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:46.468024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.468390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.468736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.488098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:46.489283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:46.506546 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:46.551067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1269/2535] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:46.560628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.560916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.561197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.580009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:46.581251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:46.589694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1267/2536] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:46.593973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.604149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a502aed-965a-4e80-a19b-48fd6fc8bef2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.608869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.609554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.609844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.610125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.610908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:46.611226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.612852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.625100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1270/2537] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:46.629429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.639298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d193fd-80d3-4e67-9ad9-f3645cca6351 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.644005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.644652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.644940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.645222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.645923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.647874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.672806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1268/2538] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:46 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:46.676547 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.687263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0abe3-c496-4d26-a355-9e7dec97e91e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.742431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.743120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.743408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.743686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.744371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.748718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.769297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1271/2539] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:46 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:46.773564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.784626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1038-b2f1-4d3d-9402-439e09f7b701 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.789437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.790095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.790525 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.790813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.791598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.793296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.807023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1269/2540] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:46.808944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.819820 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c52-ea64-4f4b-bc9c-1c58b3b37557 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.825202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634705e9-fbcb-4a16-ac19-fa4e26b699b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.825865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634705e9-fbcb-4a16-ac19-fa4e26b699b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.826153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634705e9-fbcb-4a16-ac19-fa4e26b699b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.826435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634705e9-fbcb-4a16-ac19-fa4e26b699b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.827224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634705e9-fbcb-4a16-ac19-fa4e26b699b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.828842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634705e9-fbcb-4a16-ac19-fa4e26b699b6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.839057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1272/2541] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:46.843630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:46.853815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056089-c627-4ff4-b790-f43b7a46a896 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:46.858468 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:46.859635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.859797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.860078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] PATH_INFO: `/v3/projects/a14b62a16f8149ef9d5542eb44a1ac6d/users/bd951cce59394bed870eef2a20a044e1/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.876339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14b62a16f8149ef9d5542eb44a1ac6d, user_id=bd951cce59394bed870eef2a20a044e1, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:46.879043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:46.893620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1270/2542] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:46 2022] PUT /identity/v3/projects/a14b62a16f8149ef9d5542eb44a1ac6d/users/bd951cce59394bed870eef2a20a044e1/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:46.904617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:46.904910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:46.905195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:46.932191 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] MFA Rules not processed for user `bd951cce59394bed870eef2a20a044e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:46.934124 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:47.011006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1273/2543] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:47.012560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.036263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:47.095816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36bb1d-6579-4dbb-87f9-8738a28c28c7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.101067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.101864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.102265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.102641 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.103482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.106895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.139237 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1271/2544] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:47 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:47.143422 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.155632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd435cf-033f-4097-aaa4-323d0eb74170 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.168663 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:47.212811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.213477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.213777 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.214067 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.215071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.217015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.237183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1274/2545] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:47 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:47.241302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.253117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fe08c-98ae-47f6-8201-877427b849b3 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.257844 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.258495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.258783 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.259098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.259817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.261564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.271993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1272/2546] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:47.276050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.286288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a290e3-004c-46c9-b179-4f43f87a71d9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.290915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b53e9-07f5-40ce-b5f6-8c87cfaa5c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.291571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b53e9-07f5-40ce-b5f6-8c87cfaa5c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.291860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b53e9-07f5-40ce-b5f6-8c87cfaa5c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.292139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b53e9-07f5-40ce-b5f6-8c87cfaa5c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.293405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b53e9-07f5-40ce-b5f6-8c87cfaa5c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.295154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b53e9-07f5-40ce-b5f6-8c87cfaa5c43 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.304980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1275/2547] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:47.309912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.319579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd7dd2-a279-4251-8dda-34c47f24b0ee admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.324444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.325453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.325740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.326035 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] PATH_INFO: `/v3/projects/aebaf24810c84e9882ed892d6ea03f0c/users/337e285a5bec475d9f93d91428edd5bd/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.341141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aebaf24810c84e9882ed892d6ea03f0c, user_id=337e285a5bec475d9f93d91428edd5bd, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.343892 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.357067 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1273/2548] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:47 2022] PUT /identity/v3/projects/aebaf24810c84e9882ed892d6ea03f0c/users/337e285a5bec475d9f93d91428edd5bd/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:47.367293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.367594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.367883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.394456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] MFA Rules not processed for user `337e285a5bec475d9f93d91428edd5bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:47.395734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:47.460517 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1276/2549] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:47.465664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.527486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdf489-ffc2-42fa-a932-2f6df7622f01 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.532101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.532753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.533039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.533321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.533999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.535964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.562691 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1274/2550] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:47 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:47.566638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.577643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca7e90-27a7-4001-91a2-d0577d2437e6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.635509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.636583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.636938 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.637269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.638006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.640008 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.661592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1277/2551] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:47 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:47.665560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.676951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fff9e9-c780-4ba8-99d8-92d74c34791e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.681639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.682364 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.682717 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.683249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.684258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.685862 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.696426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1275/2552] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:47.701308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.711378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a4b4e-8474-4935-a155-ffa197ce2d7a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.716780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.717653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.717942 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.718224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] PATH_INFO: `/v3/projects/611f0f5909d4470a9cec78dcf2754d3c/users/883df7511e8d411a9bda63505585f2bb/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.730483 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:47.733475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611f0f5909d4470a9cec78dcf2754d3c, user_id=883df7511e8d411a9bda63505585f2bb, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.736308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.749272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1278/2553] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:47 2022] PUT /identity/v3/projects/611f0f5909d4470a9cec78dcf2754d3c/users/883df7511e8d411a9bda63505585f2bb/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:47.753247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.763851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f433be-2837-46af-8de2-5ad8b542899a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.805245 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:47.821837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454f6d0-bde8-44dd-8d02-2467a6f70c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.822441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454f6d0-bde8-44dd-8d02-2467a6f70c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.822732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454f6d0-bde8-44dd-8d02-2467a6f70c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.823048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454f6d0-bde8-44dd-8d02-2467a6f70c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.823772 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454f6d0-bde8-44dd-8d02-2467a6f70c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.825480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454f6d0-bde8-44dd-8d02-2467a6f70c4e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.835329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1276/2554] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:47.839401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.849935 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5f76f-9631-4d3c-82ea-f6df39a28c35 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:47.854536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:47.855975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.856259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.856538 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] PATH_INFO: `/v3/projects/611f0f5909d4470a9cec78dcf2754d3c/users/883df7511e8d411a9bda63505585f2bb/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.858514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611f0f5909d4470a9cec78dcf2754d3c, user_id=883df7511e8d411a9bda63505585f2bb, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:47.861214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:47.872404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1279/2555] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:47 2022] PUT /identity/v3/projects/611f0f5909d4470a9cec78dcf2754d3c/users/883df7511e8d411a9bda63505585f2bb/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:47.883463 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67638914-7580-486e-8cfd-12dfab66497d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:47.883768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67638914-7580-486e-8cfd-12dfab66497d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:47.884084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67638914-7580-486e-8cfd-12dfab66497d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:47.909875 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-67638914-7580-486e-8cfd-12dfab66497d None None] MFA Rules not processed for user `883df7511e8d411a9bda63505585f2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:47.911121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-67638914-7580-486e-8cfd-12dfab66497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:47.979263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1277/2556] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:47.991967 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:47.993668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:48.052757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf8805-163a-4c6d-b24e-b6a68bace634 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:48.056371 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:48.057033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:48.057308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:48.057592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:48.114057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:48.115473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:48.124344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1280/2557] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:49.633311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67638914-7580-486e-8cfd-12dfab66497d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:49.635312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-67638914-7580-486e-8cfd-12dfab66497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:49.647354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67638914-7580-486e-8cfd-12dfab66497d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:49.650904 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:49.652042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:49.652337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:49.652759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:49.710085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:49.711479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:49.720791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1278/2558] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:50.237298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:50.246119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c6674-a4f2-4649-8f6f-c33e25d9b1c9 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:50.249791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541969e1-80dd-4827-ae60-bc4487183d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:50.250444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541969e1-80dd-4827-ae60-bc4487183d79 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:50.250738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541969e1-80dd-4827-ae60-bc4487183d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:50.251061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541969e1-80dd-4827-ae60-bc4487183d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:50.260212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541969e1-80dd-4827-ae60-bc4487183d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:50.262122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541969e1-80dd-4827-ae60-bc4487183d79 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:50.271512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1281/2559] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:54.157113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:54.226440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8338-eb8c-47e9-a951-05f2b8a1a2ec service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:54.232398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:54.233306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:54.233597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:54.233881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] PATH_INFO: `/v3/users/bd951cce59394bed870eef2a20a044e1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:54.235172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd951cce59394bed870eef2a20a044e1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:54.237151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:54.243730 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:54.288205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd951cce59394bed870eef2a20a044e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.288851 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd951cce59394bed870eef2a20a044e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.289452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd951cce59394bed870eef2a20a044e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.297040 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd951cce59394bed870eef2a20a044e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.308922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd951cce59394bed870eef2a20a044e1'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.321880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1279/2560] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:54 2022] DELETE /identity/v3/users/bd951cce59394bed870eef2a20a044e1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:54.326674 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:54.327174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:54.327291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:54.346069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:54.347325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:54.409747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1282/2561] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:54.420079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:54.421512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:54.479622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c38e30-e7e1-49d3-9f36-7970c01d3558 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:54.483315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:54.483962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:54.484251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:54.484534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:54.493481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:54.494935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:54.504958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1280/2562] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:54.779401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:54.780846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:54.794022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaab737-8493-4f89-84dd-d2a529ca200e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:54.798918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:54.799941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:54.800229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:54.800514 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] PATH_INFO: `/v3/projects/a14b62a16f8149ef9d5542eb44a1ac6d` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:54.801827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14b62a16f8149ef9d5542eb44a1ac6d)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:54.803851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:54.821265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:54.855561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14b62a16f8149ef9d5542eb44a1ac6d', 'request_id': 'req-fb00a0be-51b4-4133-95af-82c73ddf2973'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.862195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14b62a16f8149ef9d5542eb44a1ac6d', 'request_id': 'req-fb00a0be-51b4-4133-95af-82c73ddf2973'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.871150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14b62a16f8149ef9d5542eb44a1ac6d', 'request_id': 'req-fb00a0be-51b4-4133-95af-82c73ddf2973'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.883258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] The token cache is being invalidate because project a14b62a16f8149ef9d5542eb44a1ac6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:54.883551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.884196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:54.885842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1283/2563] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:54 2022] DELETE /identity/v3/projects/a14b62a16f8149ef9d5542eb44a1ac6d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:54.890475 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:54.900773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db02c0-5201-4989-a6c6-687ead15ad5b service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:54.970985 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:54.971872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:54.972155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:54.972432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] PATH_INFO: `/v3/users/337e285a5bec475d9f93d91428edd5bd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:54.973626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=337e285a5bec475d9f93d91428edd5bd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:54.975603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:55.023287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:55.053123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '337e285a5bec475d9f93d91428edd5bd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.053794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '337e285a5bec475d9f93d91428edd5bd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.054438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '337e285a5bec475d9f93d91428edd5bd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.072652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '337e285a5bec475d9f93d91428edd5bd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.085867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '337e285a5bec475d9f93d91428edd5bd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.100393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1281/2564] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:54 2022] DELETE /identity/v3/users/337e285a5bec475d9f93d91428edd5bd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:55.460650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:55.462353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:55.477005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00a0be-51b4-4133-95af-82c73ddf2973 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:55.543159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:55.544465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:55.544797 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:55.545199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] PATH_INFO: `/v3/projects/aebaf24810c84e9882ed892d6ea03f0c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:55.546879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aebaf24810c84e9882ed892d6ea03f0c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:55.549365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:55.609150 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aebaf24810c84e9882ed892d6ea03f0c', 'request_id': 'req-71470201-6679-46c2-a1eb-215c88ca41f0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.617254 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aebaf24810c84e9882ed892d6ea03f0c', 'request_id': 'req-71470201-6679-46c2-a1eb-215c88ca41f0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.625636 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aebaf24810c84e9882ed892d6ea03f0c', 'request_id': 'req-71470201-6679-46c2-a1eb-215c88ca41f0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.638553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] The token cache is being invalidate because project aebaf24810c84e9882ed892d6ea03f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:55.638874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.639657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.641204 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1284/2565] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:55 2022] DELETE /identity/v3/projects/aebaf24810c84e9882ed892d6ea03f0c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:55.645365 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:55.646906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:55.660241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec2eaa-0026-41b3-ba7d-e31379d23e16 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:55.716751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:55.717622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:55.717915 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:55.718198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] PATH_INFO: `/v3/users/883df7511e8d411a9bda63505585f2bb` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:55.719690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883df7511e8d411a9bda63505585f2bb)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:55.721716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:55.766144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883df7511e8d411a9bda63505585f2bb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.766802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883df7511e8d411a9bda63505585f2bb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.767437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883df7511e8d411a9bda63505585f2bb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.774615 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883df7511e8d411a9bda63505585f2bb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.787223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c97d9804-fe73-4700-a622-f5d4605b2949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883df7511e8d411a9bda63505585f2bb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:55.800682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1282/2566] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:55 2022] DELETE /identity/v3/users/883df7511e8d411a9bda63505585f2bb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:56.123902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.125510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:56.140348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71470201-6679-46c2-a1eb-215c88ca41f0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.197621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:56.223400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.224276 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.224563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.224842 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] PATH_INFO: `/v3/projects/611f0f5909d4470a9cec78dcf2754d3c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.226181 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=611f0f5909d4470a9cec78dcf2754d3c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.228228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.287506 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611f0f5909d4470a9cec78dcf2754d3c', 'request_id': 'req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:56.294120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611f0f5909d4470a9cec78dcf2754d3c', 'request_id': 'req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:56.302140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611f0f5909d4470a9cec78dcf2754d3c', 'request_id': 'req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:56.313890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] The token cache is being invalidate because project 611f0f5909d4470a9cec78dcf2754d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:56.314184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:56.314948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-b86e2f41-9346-45ca-a1ce-43b3ccc863a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:56.316616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1285/2567] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:56 2022] DELETE /identity/v3/projects/611f0f5909d4470a9cec78dcf2754d3c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:56.321628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.321922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.322229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.343925 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:56.345658 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:56.366055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:56.413289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1283/2568] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:56.422093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.422304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.422590 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.443494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:56.445075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:56.464417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1286/2569] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:56.466213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.479501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16869eff-ed50-4ea8-abce-c704d94af8ef None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.484458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.485126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.485472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.485759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.486485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:56.486791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.488443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.502608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1284/2570] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:14:56.508057 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.519878 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aec52a-208b-4656-981e-83775ad48e58 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.524817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.525488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.525775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.526056 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.526800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.528945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.561582 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1287/2571] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:56 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:56.563262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.574766 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906aeb-5bed-40a6-a279-c428b7cdf529 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.648690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.649388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.649636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.650328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.651189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.653384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.675436 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1285/2572] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:56 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:56.679467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.690819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f02a-d3fd-446f-9387-b9e823cd3614 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.695746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.696430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.696775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.697062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.697794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.700005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.717231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1288/2573] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:56.718115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.728796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3815d-cac9-4dcf-9d22-8a2920724d0e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.734513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.735217 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.735508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.735786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.736512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.738272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.751063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1286/2574] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:14:56.754973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.757516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.759715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:56.767796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132d7e6-0202-4310-9d25-30c769b103f9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.773015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82a30-2d2b-458d-a946-4ee8cfc2ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.773991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82a30-2d2b-458d-a946-4ee8cfc2ab89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.774310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82a30-2d2b-458d-a946-4ee8cfc2ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.774640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82a30-2d2b-458d-a946-4ee8cfc2ab89 admin admin] PATH_INFO: `/v3/projects/f1ddc4000ca0460da1132c31f18838ea/users/b30d4342242840b18608e597cf27947b/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.796790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82a30-2d2b-458d-a946-4ee8cfc2ab89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ddc4000ca0460da1132c31f18838ea, user_id=b30d4342242840b18608e597cf27947b, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:56.799853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82a30-2d2b-458d-a946-4ee8cfc2ab89 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:56.819086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1289/2575] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:56 2022] PUT /identity/v3/projects/f1ddc4000ca0460da1132c31f18838ea/users/b30d4342242840b18608e597cf27947b/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:56.830999 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.831308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.831572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:56.862239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1934ede-9265-4b27-a5e6-6a86b25326f6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:56.865745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] MFA Rules not processed for user `b30d4342242840b18608e597cf27947b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:56.866496 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:56.919248 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:56.928617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:56.952507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1290/2576] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:56.964771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:56.967475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:56.982472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:56.983538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:56.984105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:56.984493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.004239 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:57.031743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:57.033814 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:57.048850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1287/2577] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:57.056217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a90b-0564-4fc5-9024-ea20edd20c91 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:57.061156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeecb0-e0a5-4a6b-9c12-53118fe4144b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:57.061965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeecb0-e0a5-4a6b-9c12-53118fe4144b service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:57.062326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeecb0-e0a5-4a6b-9c12-53118fe4144b service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:57.062669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeecb0-e0a5-4a6b-9c12-53118fe4144b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.074521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeecb0-e0a5-4a6b-9c12-53118fe4144b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:57.077174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeecb0-e0a5-4a6b-9c12-53118fe4144b service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:57.093606 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1291/2578] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:57.119820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:57.184930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae358-f9d1-4f7f-b568-238be6b7c391 service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:57.189366 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:57.190242 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:57.190528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:57.190806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] PATH_INFO: `/v3/users/b30d4342242840b18608e597cf27947b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.192044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30d4342242840b18608e597cf27947b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:57.193974 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:57.245580 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30d4342242840b18608e597cf27947b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.246454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30d4342242840b18608e597cf27947b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.247196 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30d4342242840b18608e597cf27947b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.254679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30d4342242840b18608e597cf27947b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.267871 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30d4342242840b18608e597cf27947b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.283487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1288/2579] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:14:57 2022] DELETE /identity/v3/users/b30d4342242840b18608e597cf27947b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:57.287883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:57.288293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:57.288640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.307609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:57.309009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:57.387259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1292/2580] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:57.396937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:57.398549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:57.436708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:57.462476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70277bbf-abb5-4f8d-9b47-ef1c44e61467 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:57.467303 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362db79-4d51-4bcd-9093-8bd9c34e77d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:57.468068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362db79-4d51-4bcd-9093-8bd9c34e77d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:57.468597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362db79-4d51-4bcd-9093-8bd9c34e77d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:57.469069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362db79-4d51-4bcd-9093-8bd9c34e77d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.481991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362db79-4d51-4bcd-9093-8bd9c34e77d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:57.483540 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362db79-4d51-4bcd-9093-8bd9c34e77d5 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:57.501980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1289/2581] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:14:57.812868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:57.814508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:57.834179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58577135-66a4-4611-bb35-e6206ba2a703 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:57.839516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:57.840718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:57.841072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:57.841428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] PATH_INFO: `/v3/projects/f1ddc4000ca0460da1132c31f18838ea` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.843269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ddc4000ca0460da1132c31f18838ea)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:57.846139 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:57.907802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ddc4000ca0460da1132c31f18838ea', 'request_id': 'req-05165b9f-e5f5-4940-8a76-cf29b9605d6a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.915286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ddc4000ca0460da1132c31f18838ea', 'request_id': 'req-05165b9f-e5f5-4940-8a76-cf29b9605d6a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.922769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ddc4000ca0460da1132c31f18838ea', 'request_id': 'req-05165b9f-e5f5-4940-8a76-cf29b9605d6a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.933482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] The token cache is being invalidate because project f1ddc4000ca0460da1132c31f18838ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:14:57.933922 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.934751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-05165b9f-e5f5-4940-8a76-cf29b9605d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:14:57.936466 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1293/2582] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:14:57 2022] DELETE /identity/v3/projects/f1ddc4000ca0460da1132c31f18838ea => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:14:57.942922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:57.943398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:57.943885 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:57.963852 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:57.965113 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:58.024602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1290/2583] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:58.097912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.098199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.098477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.117476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:58.118671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:58.128078 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1294/2584] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:58.132452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.143230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb294e-b160-4a59-a6bf-df05de58bc3b None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.147940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.148592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.148877 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.149158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.149880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:14:58.150186 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.151949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.163438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1291/2585] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:14:58.168064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.177154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4f96f-caa4-4af1-8456-2efa775f6748 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.181909 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.182563 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.183488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.183784 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.184469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.186426 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.190540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:58.214786 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1295/2586] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:14:58 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:14:58.218950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.229085 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907842-f729-4b48-ad16-f0ea1977af5e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.285629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.286385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.286583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.286872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.287571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.289516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.310287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1292/2587] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:14:58 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:14:58.316691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.334224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969e9c5-aa0c-49dc-a397-e4ba2d2499d8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.341770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.342907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.343409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.343900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.345473 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.348453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.366522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1296/2588] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:58.371304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.381261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f71f-ebd4-4fed-b55e-927a8d405dc6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.386134 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.386791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.387131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.387390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.388145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.389816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.390704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.392472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:58.403244 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1293/2589] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:14:58.404293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.414098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17e0c4-4940-4ba0-ab67-32854b260a48 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.421618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc817-15bf-4322-8760-b2658bb916dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.422504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc817-15bf-4322-8760-b2658bb916dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.422794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc817-15bf-4322-8760-b2658bb916dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.423123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc817-15bf-4322-8760-b2658bb916dd admin admin] PATH_INFO: `/v3/projects/621a97bcf68f4dea8bce2be900c2eefd/users/8c4a7545593a45dca9c0029d29910811/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.446757 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc817-15bf-4322-8760-b2658bb916dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=621a97bcf68f4dea8bce2be900c2eefd, user_id=8c4a7545593a45dca9c0029d29910811, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.451527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc817-15bf-4322-8760-b2658bb916dd admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.474582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1294/2590] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:14:58 2022] PUT /identity/v3/projects/621a97bcf68f4dea8bce2be900c2eefd/users/8c4a7545593a45dca9c0029d29910811/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:14:58.479569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.480640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.480932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.488815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed399eed-0f9e-4852-892a-1fed36bf4f09 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.498234 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:58.499473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:58.511906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:58.553342 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.554243 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.554719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.555054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.556829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:58.563297 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1295/2591] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:14:58.573238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:58.574802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:58.642003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca04f7e-f44c-4b1a-9716-fd6322f688d6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:58.645016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:14:58.646209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5077052-9643-4ee9-b494-7acee93b4f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:58.647071 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5077052-9643-4ee9-b494-7acee93b4f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:58.647333 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5077052-9643-4ee9-b494-7acee93b4f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:58.647560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5077052-9643-4ee9-b494-7acee93b4f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:58.658347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5077052-9643-4ee9-b494-7acee93b4f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.659579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:58.659847 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5077052-9643-4ee9-b494-7acee93b4f25 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.661219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:58.671123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1296/2592] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:14:58.689469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1297/2593] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:14:59.519259 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb921-1ba6-4fac-a7b0-a0d2b688a302 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:59.520056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb921-1ba6-4fac-a7b0-a0d2b688a302 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:59.520371 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb921-1ba6-4fac-a7b0-a0d2b688a302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:59.554001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-10ebb921-1ba6-4fac-a7b0-a0d2b688a302 None None] MFA Rules not processed for user `8c4a7545593a45dca9c0029d29910811`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:14:59.555687 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-10ebb921-1ba6-4fac-a7b0-a0d2b688a302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:59.652965 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1297/2594] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:14:59.671327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:14:59.673607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:14:59.776350 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce75ef2-08fe-436c-98a4-9732c4b5f58c service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:14:59.782707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:14:59.783854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:14:59.784306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:14:59.784747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:14:59.800491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:14:59.802987 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:14:59.816178 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1298/2595] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:14:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:02.505588 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:02.506101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:02.506225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:02.524405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] MFA Rules not processed for user `8c4a7545593a45dca9c0029d29910811`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:02.525623 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:02.532988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1298/2596] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:02.765891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:02.775456 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd511f4-217c-4063-a665-1012e9e1e3c8 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:02.779389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:02.780118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:02.780416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:02.780779 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:02.782490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:02.793526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:02.795034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:02.803902 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1299/2597] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:03.130115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:03.131569 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:03.182572 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:03.206913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10dae3-927f-47c0-bcb3-b3934cdc9501 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:03.211907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:03.212557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:03.212849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:03.213139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:03.214995 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:03.226869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:03.228460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:03.238452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1299/2598] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:09.376844 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:09.378356 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:09.389502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e97150-a893-45de-845d-49b3d6104bea service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:09.392755 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:09.393407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:09.393696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:09.393978 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:09.395800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:09.410200 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:09.412549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:09.423752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1300/2599] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:13.161667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:13.163184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:13.179992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaccb0-5503-4b61-b614-ec945b21298a service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:13.186483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:13.187639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.187931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.188378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] PATH_INFO: `/v3/users/c8bda0c079054e198ffeca4c0ac18bf2` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.189836 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8bda0c079054e198ffeca4c0ac18bf2)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:13.192606 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:13.241193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8bda0c079054e198ffeca4c0ac18bf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.241963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8bda0c079054e198ffeca4c0ac18bf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.242656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8bda0c079054e198ffeca4c0ac18bf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.250510 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8bda0c079054e198ffeca4c0ac18bf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.262752 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b96305cf-101d-4a33-827c-c901d077c40e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8bda0c079054e198ffeca4c0ac18bf2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.275954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1300/2600] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:13 2022] DELETE /identity/v3/users/c8bda0c079054e198ffeca4c0ac18bf2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:13.464004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:13.474749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e04f5-eaf7-430f-86ba-238df759db8c service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:13.538252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:13.539166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.539453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.539734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] PATH_INFO: `/v3/projects/e4e76a3125df4436abf5f788e35d492f` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.540931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e76a3125df4436abf5f788e35d492f)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:13.543650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:13.595490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e76a3125df4436abf5f788e35d492f', 'request_id': 'req-efb034af-f87f-44f2-ada1-1fc597f054f3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.602167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e76a3125df4436abf5f788e35d492f', 'request_id': 'req-efb034af-f87f-44f2-ada1-1fc597f054f3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.613099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e76a3125df4436abf5f788e35d492f', 'request_id': 'req-efb034af-f87f-44f2-ada1-1fc597f054f3'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.614993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:13.632370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] The token cache is being invalidate because project e4e76a3125df4436abf5f788e35d492f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:13.633110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.633801 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-efb034af-f87f-44f2-ada1-1fc597f054f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:13.636026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1301/2601] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:13 2022] DELETE /identity/v3/projects/e4e76a3125df4436abf5f788e35d492f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:13.640934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.641240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.641537 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.660942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:13.662263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:13.729975 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1301/2602] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:13.738915 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.739216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.739507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.758365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:13.759930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:13.769432 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1302/2603] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:13.773856 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:13.784080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e91406-388e-4eb3-92ea-927d5251515e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:13.788970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:13.789666 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.790017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.790328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.791099 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:13.791416 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:13.793660 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:13.806452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1302/2604] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:15:13.811643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:13.822651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eedfa4-374d-4502-8f96-984b7afc4081 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:13.827759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:13.828585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.828875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.829153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.829859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:13.831841 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:13.865360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1303/2605] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:13 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:13.869834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:13.881125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f2474-61d0-4c91-83ea-3be793722f3d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:13.912685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:13.945460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:13.946141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.946431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.946714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.947476 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:13.949927 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:13.971334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1303/2606] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:13 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:13.975214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:13.986174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7f236-6656-44cf-b4b3-92b9221789ce admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:13.990750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:13.991455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:13.991744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:13.992023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:13.992740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:13.994355 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.005550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1304/2607] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:14.009532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.020486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5722776-3e24-478d-ae45-96d44536310d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.025421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f7422-7445-411f-8a4a-19405d7e86c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.026096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f7422-7445-411f-8a4a-19405d7e86c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.026374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f7422-7445-411f-8a4a-19405d7e86c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.026655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f7422-7445-411f-8a4a-19405d7e86c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.028688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f7422-7445-411f-8a4a-19405d7e86c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.030279 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f7422-7445-411f-8a4a-19405d7e86c9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.040622 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1304/2608] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:14.045183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.055281 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd9fac-ec25-4069-bd71-51596d955f76 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.060578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.061453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.061736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.062016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] PATH_INFO: `/v3/projects/f08efe90d70f400a99bc131076bd52e3/users/d9d19dbd241142df9e5d819616efa3d6/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.078269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08efe90d70f400a99bc131076bd52e3, user_id=d9d19dbd241142df9e5d819616efa3d6, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.080993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.094883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1305/2609] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:14 2022] PUT /identity/v3/projects/f08efe90d70f400a99bc131076bd52e3/users/d9d19dbd241142df9e5d819616efa3d6/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:14.106787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.107050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.107512 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.135480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] MFA Rules not processed for user `d9d19dbd241142df9e5d819616efa3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:14.137249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:14.206549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1305/2610] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:14.211280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.276691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6271-2ac5-44d4-8a29-bf70cd21ca0f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.282550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.283529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.283633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.283917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.284612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.286655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.289140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.291314 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:14.335571 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1306/2611] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:14 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:14.339798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.410529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e8971-1931-4e0e-9468-a54969fb01cc None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.417404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f7ca7-285c-4ee1-b207-e42015a22328 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.423031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.423722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.424009 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.424292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.425071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.427482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.445369 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:14.456177 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1307/2612] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:14 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:14.457093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:14.457230 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.468662 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bac43-50fe-42dd-af28-b192a83bce93 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.473966 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.474704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.475023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.475306 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.476033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.477647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.485014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.485677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.485970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.486255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.488054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1308/2613] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:14.488171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:14.492232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.502027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.503459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9251e-25e9-46f5-a27b-5d6f57787b35 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.503653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.508131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.509014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.509299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.509583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] PATH_INFO: `/v3/projects/2e674c93e3d44cac93138a9c6b4109a7/users/63abb68d896c479d8eeeba91e6dd461b/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.514832 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1306/2614] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:14.525269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e674c93e3d44cac93138a9c6b4109a7, user_id=63abb68d896c479d8eeeba91e6dd461b, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.528733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.541975 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1309/2615] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:14 2022] PUT /identity/v3/projects/2e674c93e3d44cac93138a9c6b4109a7/users/63abb68d896c479d8eeeba91e6dd461b/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:14.546808 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.558916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde8095-251a-42cb-86f2-0fa08ac1331e service nova] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.616487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131e6d7-0895-436d-8ea6-bfed59c2ae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.617082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e6d7-0895-436d-8ea6-bfed59c2ae5c admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.617376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e6d7-0895-436d-8ea6-bfed59c2ae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.617664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131e6d7-0895-436d-8ea6-bfed59c2ae5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.618397 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e6d7-0895-436d-8ea6-bfed59c2ae5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.620057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131e6d7-0895-436d-8ea6-bfed59c2ae5c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.630652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1307/2616] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:14.634868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.648470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a28075-4e22-4cc1-bd38-b33319bdfbeb admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.661353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.664107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.664640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.665086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] PATH_INFO: `/v3/projects/2e674c93e3d44cac93138a9c6b4109a7/users/63abb68d896c479d8eeeba91e6dd461b/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.668222 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e674c93e3d44cac93138a9c6b4109a7, user_id=63abb68d896c479d8eeeba91e6dd461b, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.671051 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.682022 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1310/2617] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:14 2022] PUT /identity/v3/projects/2e674c93e3d44cac93138a9c6b4109a7/users/63abb68d896c479d8eeeba91e6dd461b/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:14.693788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.694619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.694619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.723377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] MFA Rules not processed for user `63abb68d896c479d8eeeba91e6dd461b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:14.724624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:14.802556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1308/2618] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:14.814884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.816605 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:14.878929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f568bc-5098-4918-9205-32f372f416c8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.883059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.883721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.884010 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.884294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:14.894259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:14.895747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:14.906196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1311/2619] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:14.978285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:14.988640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe147d2-189e-46be-8bb3-36a1ffea26f9 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:14.992288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa73f9-bc6e-442b-ae4b-a2bd219cd3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:14.992926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa73f9-bc6e-442b-ae4b-a2bd219cd3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:14.993216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa73f9-bc6e-442b-ae4b-a2bd219cd3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:14.993498 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa73f9-bc6e-442b-ae4b-a2bd219cd3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:15.038872 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=100 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:15.056933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa73f9-bc6e-442b-ae4b-a2bd219cd3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:15.059431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa73f9-bc6e-442b-ae4b-a2bd219cd3b6 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:15.071567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1309/2620] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:24.901323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:25.064624 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=7 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:28.258226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:28.259672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:28.334250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdafce-3350-4f41-afb6-e7980d0c3e87 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:28.339905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:28.341093 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:28.341395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:28.341680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] PATH_INFO: `/v3/users/f0c93a9b14af4e40ac2d93ee31e1697e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:28.343170 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c93a9b14af4e40ac2d93ee31e1697e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:28.345158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:28.402973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c93a9b14af4e40ac2d93ee31e1697e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:28.403984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c93a9b14af4e40ac2d93ee31e1697e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:28.411257 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c93a9b14af4e40ac2d93ee31e1697e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:28.411896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c93a9b14af4e40ac2d93ee31e1697e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:28.420675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c93a9b14af4e40ac2d93ee31e1697e'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:28.440824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1312/2621] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:28 2022] DELETE /identity/v3/users/f0c93a9b14af4e40ac2d93ee31e1697e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:28.445423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:28.445903 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:28.446018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:28.463267 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:28.464486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:28.532570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1310/2622] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:28.543614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:28.546212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:28.608028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503b1e2-5f65-4cd2-8fb3-e19450da0e3c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:28.611722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:28.612368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:28.612653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:28.612932 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:28.622918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:28.624382 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:28.634096 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1313/2623] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:28.922097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:28.923576 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:28.935526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971fa4f-a77a-48e4-8642-54c83e8cfc6f None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:28.940448 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:28.941323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:28.941601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:28.941966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] PATH_INFO: `/v3/projects/52cb240a8a0f4174b55b2f31c8984601` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:28.943168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52cb240a8a0f4174b55b2f31c8984601)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:28.945073 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.015125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52cb240a8a0f4174b55b2f31c8984601', 'request_id': 'req-a57acfdc-5fdb-47a6-b745-755c16992653'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.021789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52cb240a8a0f4174b55b2f31c8984601', 'request_id': 'req-a57acfdc-5fdb-47a6-b745-755c16992653'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.028158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52cb240a8a0f4174b55b2f31c8984601', 'request_id': 'req-a57acfdc-5fdb-47a6-b745-755c16992653'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.041857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] The token cache is being invalidate because project 52cb240a8a0f4174b55b2f31c8984601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:29.042153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.042782 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.044620 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1311/2624] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:28 2022] DELETE /identity/v3/projects/52cb240a8a0f4174b55b2f31c8984601 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:29.048650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.058495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04ef78-f0c9-427e-9d90-b8440ed34d12 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.086633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:29.131302 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.132203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.132519 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.132763 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] PATH_INFO: `/v3/users/26c490a3010940b99146e1878c183589` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.133944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c490a3010940b99146e1878c183589)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.136205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.201764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c490a3010940b99146e1878c183589'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.202628 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c490a3010940b99146e1878c183589'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.210285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c490a3010940b99146e1878c183589'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.211151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c490a3010940b99146e1878c183589'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.211494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.213171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.219903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-77d1cbac-862c-485f-9f40-4e1f4e8e37dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c490a3010940b99146e1878c183589'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.227718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57acfdc-5fdb-47a6-b745-755c16992653 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.232960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.233936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.234299 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.234651 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] PATH_INFO: `/v3/users/d9d19dbd241142df9e5d819616efa3d6` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.236094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d19dbd241142df9e5d819616efa3d6)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.238172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.247832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1314/2625] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:29 2022] DELETE /identity/v3/users/26c490a3010940b99146e1878c183589 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:29.290902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d19dbd241142df9e5d819616efa3d6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.291604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d19dbd241142df9e5d819616efa3d6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.292528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d19dbd241142df9e5d819616efa3d6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.301042 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d19dbd241142df9e5d819616efa3d6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.323057 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d19dbd241142df9e5d819616efa3d6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.328038 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:29.340377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1312/2626] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:29 2022] DELETE /identity/v3/users/d9d19dbd241142df9e5d819616efa3d6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:29.345146 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.345958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.345958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.364530 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:29.365888 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.444892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1315/2627] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:29.455688 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.457197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.522671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678893a-6ece-4460-ad3a-c3c58824819a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.526461 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c203c-fd17-4f32-8225-df48060ae2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.527145 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c203c-fd17-4f32-8225-df48060ae2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.527432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c203c-fd17-4f32-8225-df48060ae2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.527712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c203c-fd17-4f32-8225-df48060ae2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.541671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.541850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c203c-fd17-4f32-8225-df48060ae2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.543164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c203c-fd17-4f32-8225-df48060ae2a2 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.543286 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.554604 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1313/2628] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:29.562498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081dcda-3c33-4ca2-8111-82ba989c4ab7 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.571068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.572084 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.572475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.572957 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] PATH_INFO: `/v3/projects/0b0b1d61310b4ad5831add4f053d3041` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.574180 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0b1d61310b4ad5831add4f053d3041)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.577021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.625484 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:29.638308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0b1d61310b4ad5831add4f053d3041', 'request_id': 'req-81f9d86f-0140-419e-8bb7-976a2506d305'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.647446 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0b1d61310b4ad5831add4f053d3041', 'request_id': 'req-81f9d86f-0140-419e-8bb7-976a2506d305'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.660832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0b1d61310b4ad5831add4f053d3041', 'request_id': 'req-81f9d86f-0140-419e-8bb7-976a2506d305'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.674019 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] The token cache is being invalidate because project 0b0b1d61310b4ad5831add4f053d3041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:29.674623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.675012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-81f9d86f-0140-419e-8bb7-976a2506d305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.676680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1316/2629] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:29 2022] DELETE /identity/v3/projects/0b0b1d61310b4ad5831add4f053d3041 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:29.681464 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.681758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.682039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.701642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:29.702899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.776474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1314/2630] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:29.784494 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.784749 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.785032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.807808 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:29.811288 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.819621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1317/2631] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:29.824062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.833990 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e99f-b2fa-4cd1-a373-6c9de3a2d7b1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.839127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.839790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.840077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.840359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.841243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:29.841774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.843521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.861562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1315/2632] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:15:29.862685 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.864151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.865742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:29.879644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361181e2-4a3b-4a11-9fec-f540470e8e88 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.880817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538de8c7-9660-4d0d-9b6f-c6bf19a8aee5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:29.887388 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.888168 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.888513 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.888753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:29.888948 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.889632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:29.889760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.889932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:29.890197 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] PATH_INFO: `/v3/projects/f08efe90d70f400a99bc131076bd52e3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:29.891430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08efe90d70f400a99bc131076bd52e3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:29.891619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.893339 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:29.925219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1318/2633] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:29 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:29.930007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.953473 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08efe90d70f400a99bc131076bd52e3', 'request_id': 'req-b655118b-7f32-4344-9ca7-88ed4e4340e6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.959323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08efe90d70f400a99bc131076bd52e3', 'request_id': 'req-b655118b-7f32-4344-9ca7-88ed4e4340e6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.966636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08efe90d70f400a99bc131076bd52e3', 'request_id': 'req-b655118b-7f32-4344-9ca7-88ed4e4340e6'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.979998 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] The token cache is being invalidate because project f08efe90d70f400a99bc131076bd52e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:29.980290 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.981005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:29.983881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1316/2634] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:29 2022] DELETE /identity/v3/projects/f08efe90d70f400a99bc131076bd52e3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:29.990187 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:29.991773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.003521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:30.011907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1a0e3-8b37-43c1-ba94-49417c13e678 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.072209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655118b-7f32-4344-9ca7-88ed4e4340e6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.076912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.077787 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.078076 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.078355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] PATH_INFO: `/v3/users/63abb68d896c479d8eeeba91e6dd461b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.079596 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63abb68d896c479d8eeeba91e6dd461b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.080806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.081474 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.081579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.081771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.082028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.082824 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.084876 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.109481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1319/2635] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:29 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:30.113625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.115457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.128968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704e922-7742-4dcd-b33a-92a5449f87a2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.133676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.134339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.134624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.134950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.135669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.136386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63abb68d896c479d8eeeba91e6dd461b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.137058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63abb68d896c479d8eeeba91e6dd461b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.137303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.138565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63abb68d896c479d8eeeba91e6dd461b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.146607 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63abb68d896c479d8eeeba91e6dd461b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.149499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1320/2636] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:30.153617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.155906 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.161285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63abb68d896c479d8eeeba91e6dd461b'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.169620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059cca8-ed9e-4238-bfcf-b8250238b933 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.180794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a43b9-9968-4555-8296-d099ec2b21bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.181527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a43b9-9968-4555-8296-d099ec2b21bf admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.182447 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1317/2637] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:29 2022] DELETE /identity/v3/users/63abb68d896c479d8eeeba91e6dd461b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:30.186094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a43b9-9968-4555-8296-d099ec2b21bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.186094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a43b9-9968-4555-8296-d099ec2b21bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.186094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a43b9-9968-4555-8296-d099ec2b21bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.188595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a43b9-9968-4555-8296-d099ec2b21bf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.202721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1321/2638] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:30.207213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.285270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d2426-5ce5-4f19-9c49-3c9822c9ae5f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.291207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.292167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.293690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.293690 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] PATH_INFO: `/v3/projects/2836a216382e47ed98b39abe6c7f369e/users/8179beb79bc042daa7e1c421492233ce/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.320630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2836a216382e47ed98b39abe6c7f369e, user_id=8179beb79bc042daa7e1c421492233ce, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.323555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.350280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1318/2639] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:30 2022] PUT /identity/v3/projects/2836a216382e47ed98b39abe6c7f369e/users/8179beb79bc042daa7e1c421492233ce/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:30.365215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.365707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.365912 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.388672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:30.400809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] MFA Rules not processed for user `8179beb79bc042daa7e1c421492233ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:30.402386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.479185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1322/2640] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:30.484773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.499549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:30.515712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.517659 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.572664 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8c177-c00a-4263-931a-e3cf43978720 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.577802 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.578445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.578755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.579097 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.579810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.582051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.595267 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac69d8-2dc4-4aa5-8245-a5193567841c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.600501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.601397 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.601679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.602046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] PATH_INFO: `/v3/projects/2e674c93e3d44cac93138a9c6b4109a7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.603285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e674c93e3d44cac93138a9c6b4109a7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.605274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.615490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1319/2641] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:30.619601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.658273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e674c93e3d44cac93138a9c6b4109a7', 'request_id': 'req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.672105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e674c93e3d44cac93138a9c6b4109a7', 'request_id': 'req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.683642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e674c93e3d44cac93138a9c6b4109a7', 'request_id': 'req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.699734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] The token cache is being invalidate because project 2e674c93e3d44cac93138a9c6b4109a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:30.700269 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.701452 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-aa23d1a9-37fc-4bc6-95e2-f5fe287ee993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:30.702810 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1323/2642] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:30 2022] DELETE /identity/v3/projects/2e674c93e3d44cac93138a9c6b4109a7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:30.707979 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b153c7-3e78-4693-a1ae-76115cc86eca None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.708312 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b153c7-3e78-4693-a1ae-76115cc86eca None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.708555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b153c7-3e78-4693-a1ae-76115cc86eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.714152 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abb7ed-8d40-4a5c-af7e-b21422a3cc0b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.729054 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-83b153c7-3e78-4693-a1ae-76115cc86eca None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:30.730387 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-83b153c7-3e78-4693-a1ae-76115cc86eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.761066 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:30.780543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.781222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.781507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.781786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.782472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.784536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.798751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1324/2643] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:30.806809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.807293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.807585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.812458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1320/2644] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/users => generated 301 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:30.816157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.817810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.826336 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:30.827598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:30.829683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8807b-2bba-4680-b597-f32bf6e76b7b admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.834193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.834882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.835171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.835409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1325/2645] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:30.835527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.836296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.837889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.839395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.852621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1321/2646] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:30.852799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1575b-30ee-4209-a8c1-1283277d7527 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.853589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.855434 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.856102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.857433 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.857840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.858488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:30.858799 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.860449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.864055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b8cc3-ebfe-48c9-b84a-fe14bf7ec81f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.868665 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.869529 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.869870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.870090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] PATH_INFO: `/v3/projects/d349bc8cf29c46b08544f34159b31ea5/users/5e33c16c917047fca77d98c62f07c193/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.875110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1326/2647] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:15:30.878629 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.887614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d349bc8cf29c46b08544f34159b31ea5, user_id=5e33c16c917047fca77d98c62f07c193, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.888460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ac21a-b29d-414c-9340-1b1306a7608e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.890458 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.893034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:30.893681 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:30.893965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:30.894247 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:30.895035 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:30.897435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:30.898981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:30.906169 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1322/2648] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:30 2022] PUT /identity/v3/projects/d349bc8cf29c46b08544f34159b31ea5/users/5e33c16c917047fca77d98c62f07c193/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:30.910117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.928366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1327/2649] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:30.932453 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:30.978307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d1ce-ae28-42ba-b03b-708f3eaeeb88 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:30.995881 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867f9c-8bf9-4e68-a744-e119313e1e21 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.009468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.010275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.010572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.010891 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.011610 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.014169 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.048807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1328/2650] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:30 2022] POST /identity/v3/users => generated 278 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:31.049969 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.052759 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.053848 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.054214 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.054502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.055266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.056907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.062197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821abf0-28bb-4233-82b5-719aa8501c56 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.066883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.067556 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.067851 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.068137 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.068866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.070490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.076132 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1323/2651] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:31.080716 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.086577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1329/2652] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:31.088359 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.092680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079b3ff-24b9-451d-8e8b-70f16e8110c1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.098501 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.099454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.099743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.100028 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] PATH_INFO: `/v3/projects/d349bc8cf29c46b08544f34159b31ea5/users/5e33c16c917047fca77d98c62f07c193/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.101305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539fba1-eac6-4dbc-8a0f-4dcb60335f3c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.102041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d349bc8cf29c46b08544f34159b31ea5, user_id=5e33c16c917047fca77d98c62f07c193, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.104762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.106006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e332e38-e859-467e-b416-dce5394e47b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.107578 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e332e38-e859-467e-b416-dce5394e47b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.108132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e332e38-e859-467e-b416-dce5394e47b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.108416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e332e38-e859-467e-b416-dce5394e47b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.109478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e332e38-e859-467e-b416-dce5394e47b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.111121 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e332e38-e859-467e-b416-dce5394e47b8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.116729 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1324/2653] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:31 2022] PUT /identity/v3/projects/d349bc8cf29c46b08544f34159b31ea5/users/5e33c16c917047fca77d98c62f07c193/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:31.122032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1330/2654] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:31.126302 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.127348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.127644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.127927 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.154640 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] MFA Rules not processed for user `5e33c16c917047fca77d98c62f07c193`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:31.155737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.195421 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08dd4c-5549-4d18-8612-73bfdb56de71 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.200535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae266913-0956-44c5-9d60-569f43eb3a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.202271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae266913-0956-44c5-9d60-569f43eb3a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.202523 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae266913-0956-44c5-9d60-569f43eb3a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.202806 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae266913-0956-44c5-9d60-569f43eb3a39 admin admin] PATH_INFO: `/v3/projects/26bf66c7d87b4aa8a95c10c4cd549f64/users/e2d026e6b68e46fda6091c84ec4ae000/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.211780 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:31.222559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae266913-0956-44c5-9d60-569f43eb3a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bf66c7d87b4aa8a95c10c4cd549f64, user_id=e2d026e6b68e46fda6091c84ec4ae000, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.227286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae266913-0956-44c5-9d60-569f43eb3a39 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.243360 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1325/2655] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:31 2022] PUT /identity/v3/projects/26bf66c7d87b4aa8a95c10c4cd549f64/users/e2d026e6b68e46fda6091c84ec4ae000/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:31.256166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.256793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.257090 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.257639 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1331/2656] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:31.274416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.275853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.277522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:31.288232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] MFA Rules not processed for user `e2d026e6b68e46fda6091c84ec4ae000`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:31.290012 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.348660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6242a-8daa-4b36-bba9-7b29a7c6ee87 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.375544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1326/2657] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:31.380147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.403012 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.405050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.405050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.405050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.453769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e41ea-ba38-4c2b-b214-e31f668e740d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.459331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.460006 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.460296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.460579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.461278 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.463226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.485769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.487405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.494598 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1327/2658] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:31 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:31.495628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.497468 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1332/2659] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:31.564773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3508c-56b3-4bc3-bac6-910024ae1333 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.569472 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.570149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.570470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.570751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.571470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.573389 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.601570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1328/2660] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:31 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:31.605730 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.617105 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613cba-7b63-4ad2-b421-42e7802cf76a service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.622713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.623815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.624196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.624511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.625212 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.626829 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.635164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.640732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1333/2661] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:31.644727 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.646323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e204c3-4495-4061-9685-c179e70c039d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.651039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.652072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.652422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.652704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] PATH_INFO: `/v3/users/8179beb79bc042daa7e1c421492233ce` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.654151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8179beb79bc042daa7e1c421492233ce)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.654832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3bcd2-6515-47a7-b434-4fe0870502b4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.657055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.660205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.661581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.662034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.662331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] PATH_INFO: `/v3/projects/40fdb72856d44b9cb4d71353fe00a843/users/dd3c6a1615e64a82afd819707f0d123d/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.671437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:31.679069 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fdb72856d44b9cb4d71353fe00a843, user_id=dd3c6a1615e64a82afd819707f0d123d, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.681759 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.695977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1334/2662] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:31 2022] PUT /identity/v3/projects/40fdb72856d44b9cb4d71353fe00a843/users/dd3c6a1615e64a82afd819707f0d123d/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:31.700598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.711096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8179beb79bc042daa7e1c421492233ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:31.711767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8179beb79bc042daa7e1c421492233ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:31.712393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8179beb79bc042daa7e1c421492233ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:31.714231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:31.720118 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8179beb79bc042daa7e1c421492233ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:31.733101 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cac94140-b6ba-4d4e-8ce9-e76ba264ed84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8179beb79bc042daa7e1c421492233ce'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:31.749639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1329/2663] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:31 2022] DELETE /identity/v3/users/8179beb79bc042daa7e1c421492233ce => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:31.754992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.754992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.755271 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.775628 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:31.776455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.783220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149841f-464b-4fdd-8210-dae55d977027 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.842419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1330/2664] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:31.852626 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.853933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.860660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.861339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.861627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.861914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.862645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.864301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.878085 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1335/2665] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:31.883013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:31.886007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.902498 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33721b79-eca8-4dc8-97f9-2d9dbc426341 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.908428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ea3a8-89df-4212-8ef1-3c3389f046b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.909365 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ea3a8-89df-4212-8ef1-3c3389f046b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.909653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ea3a8-89df-4212-8ef1-3c3389f046b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.909931 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ea3a8-89df-4212-8ef1-3c3389f046b7 admin admin] PATH_INFO: `/v3/projects/40fdb72856d44b9cb4d71353fe00a843/users/dd3c6a1615e64a82afd819707f0d123d/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.911958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ea3a8-89df-4212-8ef1-3c3389f046b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fdb72856d44b9cb4d71353fe00a843, user_id=dd3c6a1615e64a82afd819707f0d123d, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:31.914625 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ea3a8-89df-4212-8ef1-3c3389f046b7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:31.923294 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee60df5-3aa1-47a7-8deb-af340dfe364e None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:31.927095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1336/2666] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:31 2022] PUT /identity/v3/projects/40fdb72856d44b9cb4d71353fe00a843/users/dd3c6a1615e64a82afd819707f0d123d/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:31.937885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.938171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.938447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:31.965413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] MFA Rules not processed for user `dd3c6a1615e64a82afd819707f0d123d`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:31.967039 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:31.969193 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:31.979263 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:31.980091 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:31.980554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:31.981758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:32.049657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:32.051185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:32.055435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1337/2667] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:32.062019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1331/2668] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:32.067049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:32.068504 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:32.105856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:32.143475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1f020-f366-4d6b-af9c-689763e1fe2a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:32.147885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:32.148586 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:32.148879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:32.149161 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:32.169418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:32.172154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:32.185354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1338/2669] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:32.275829 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:32.288662 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93c339-196c-4716-bcf1-38b545e64e1d service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:32.292481 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:32.293159 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:32.293446 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:32.293778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:32.297556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:32.380383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:32.381799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:32.383930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:32.385911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:32.399206 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1332/2670] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:32.402410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d39ea0-2e4e-444c-99c6-0733bb6f93f9 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:32.408279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:32.409044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:32.409339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:32.409614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] PATH_INFO: `/v3/projects/2836a216382e47ed98b39abe6c7f369e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:32.410944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2836a216382e47ed98b39abe6c7f369e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:32.413090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:32.476063 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2836a216382e47ed98b39abe6c7f369e', 'request_id': 'req-8768cfc1-dbb7-4589-9516-38e6366c3cd1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.485900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2836a216382e47ed98b39abe6c7f369e', 'request_id': 'req-8768cfc1-dbb7-4589-9516-38e6366c3cd1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.498449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2836a216382e47ed98b39abe6c7f369e', 'request_id': 'req-8768cfc1-dbb7-4589-9516-38e6366c3cd1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.519836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] The token cache is being invalidate because project 2836a216382e47ed98b39abe6c7f369e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:32.519836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.520826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.523761 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1339/2671] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:32 2022] DELETE /identity/v3/projects/2836a216382e47ed98b39abe6c7f369e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:32.530560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:32.548518 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0961-6f9a-487b-833b-f34c698feb45 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:32.631991 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:32.632798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:32.633083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:32.633491 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] PATH_INFO: `/v3/users/5e33c16c917047fca77d98c62f07c193` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:32.635926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e33c16c917047fca77d98c62f07c193)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:32.638673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:32.678830 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:32.689547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e33c16c917047fca77d98c62f07c193'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.690265 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e33c16c917047fca77d98c62f07c193'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.690997 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e33c16c917047fca77d98c62f07c193'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.697897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e33c16c917047fca77d98c62f07c193'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.716923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-0d4e132a-fd32-458d-8c86-b271ac7a5b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e33c16c917047fca77d98c62f07c193'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:32.733388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1333/2672] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:32 2022] DELETE /identity/v3/users/5e33c16c917047fca77d98c62f07c193 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:33.042731 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.045144 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:33.064806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768cfc1-dbb7-4589-9516-38e6366c3cd1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.138409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.139645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.139970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.140294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] PATH_INFO: `/v3/projects/d349bc8cf29c46b08544f34159b31ea5` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.141683 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d349bc8cf29c46b08544f34159b31ea5)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.144072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.207524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd349bc8cf29c46b08544f34159b31ea5', 'request_id': 'req-036f2602-8e2c-4554-9731-870cf64281ce'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:33.217819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd349bc8cf29c46b08544f34159b31ea5', 'request_id': 'req-036f2602-8e2c-4554-9731-870cf64281ce'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:33.220646 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:33.227418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd349bc8cf29c46b08544f34159b31ea5', 'request_id': 'req-036f2602-8e2c-4554-9731-870cf64281ce'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:33.242497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] The token cache is being invalidate because project d349bc8cf29c46b08544f34159b31ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:33.242830 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:33.243577 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-036f2602-8e2c-4554-9731-870cf64281ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:33.245207 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1340/2673] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:33 2022] DELETE /identity/v3/projects/d349bc8cf29c46b08544f34159b31ea5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:33.251555 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81561f21-fa74-4411-9c74-812962ba529c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.251984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81561f21-fa74-4411-9c74-812962ba529c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.252361 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81561f21-fa74-4411-9c74-812962ba529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.273218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-81561f21-fa74-4411-9c74-812962ba529c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:33.274423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-81561f21-fa74-4411-9c74-812962ba529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:33.346907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1334/2674] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:33.355094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.355401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.355697 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.383046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:33.384460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:33.397589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1341/2675] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:33.405274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81561f21-fa74-4411-9c74-812962ba529c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.415235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81561f21-fa74-4411-9c74-812962ba529c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.423443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.424470 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.424768 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.425058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.425800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:33.426119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.427764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.443689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1335/2676] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:15:33.448018 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.458258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a29f41-e455-4c37-8975-2019a69f98b9 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.463182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.463863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.464152 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.464518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.465220 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.468015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.500547 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1342/2677] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:33 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:33.505497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.519503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc6aae-35c2-442b-94f8-ca6c96e3edc0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.587629 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.589340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.589340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.589340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.589926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.592937 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.634878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1336/2678] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:33 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:33.636135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.647677 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9646775-bc6f-41ff-a6d4-be7b26662e47 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.653263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.654049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.654334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.654686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.655476 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.657125 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.688175 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1343/2679] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:33.689950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.702786 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21091704-bcb5-41cb-b258-327694d23d79 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.710943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388da54f-155a-47c1-8676-9c09ee13e5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.711930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388da54f-155a-47c1-8676-9c09ee13e5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.712410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388da54f-155a-47c1-8676-9c09ee13e5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.712563 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388da54f-155a-47c1-8676-9c09ee13e5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.713227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388da54f-155a-47c1-8676-9c09ee13e5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.715061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388da54f-155a-47c1-8676-9c09ee13e5eb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.727579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1337/2680] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:33.734136 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:33.745596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140eebc-debf-4bb7-b587-1c7850856d37 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:33.753704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:33.755367 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.755875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.756389 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] PATH_INFO: `/v3/projects/6e95855d611b4605b3df2a24da1af95b/users/9eff1a650e7b4093851d9b44cbcdbe5e/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.783060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e95855d611b4605b3df2a24da1af95b, user_id=9eff1a650e7b4093851d9b44cbcdbe5e, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:33.786167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:33.806737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1344/2681] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:33 2022] PUT /identity/v3/projects/6e95855d611b4605b3df2a24da1af95b/users/9eff1a650e7b4093851d9b44cbcdbe5e/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:33.826318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:33.826805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:33.827311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:33.862685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] MFA Rules not processed for user `9eff1a650e7b4093851d9b44cbcdbe5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:33.863999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:33.870331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:33.945362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1338/2682] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:33.951765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.040280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ee5a-92c6-4abf-8d11-5019d1f57f0c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.049451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.050133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.050416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.050752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.052171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.054245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.087261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1345/2683] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:33 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:34.092504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.108499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afae4b-0698-473d-b6bf-6b369628e0b5 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.174270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.175151 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.175435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.175741 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.176455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.178634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.202591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1339/2684] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:34 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:34.207673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.219753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94943f8-9004-4f87-938b-0806330f3f02 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.225894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b5766-9879-4863-ba52-093425952365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.226733 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b5766-9879-4863-ba52-093425952365 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.227135 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b5766-9879-4863-ba52-093425952365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.227458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b5766-9879-4863-ba52-093425952365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.228214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b5766-9879-4863-ba52-093425952365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.229883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b5766-9879-4863-ba52-093425952365 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.249075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1346/2685] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:34.253296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.265347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8f42f-de41-4e59-b569-bc8d19e754fa admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.270168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.271379 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.271560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.272027 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] PATH_INFO: `/v3/projects/823fe41ae6e243848ab0eda54112d3b0/users/c60c22e5a7b041feb68f7e421c9bf0fd/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.291682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=823fe41ae6e243848ab0eda54112d3b0, user_id=c60c22e5a7b041feb68f7e421c9bf0fd, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.296769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.315721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1340/2686] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:34 2022] PUT /identity/v3/projects/823fe41ae6e243848ab0eda54112d3b0/users/c60c22e5a7b041feb68f7e421c9bf0fd/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:34.320274 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b5766-9879-4863-ba52-093425952365 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.333145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b5766-9879-4863-ba52-093425952365 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.398874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:34.411775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63281c4-a5bd-4ed1-99d8-85828e65a945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.412746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63281c4-a5bd-4ed1-99d8-85828e65a945 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.413348 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63281c4-a5bd-4ed1-99d8-85828e65a945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.413785 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63281c4-a5bd-4ed1-99d8-85828e65a945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.416680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63281c4-a5bd-4ed1-99d8-85828e65a945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.418595 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63281c4-a5bd-4ed1-99d8-85828e65a945 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.437670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1347/2687] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:34.439340 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.451795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750ad4f-66a0-4061-b92e-da135d0bbe31 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.458790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.460283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.460755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.461224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] PATH_INFO: `/v3/projects/823fe41ae6e243848ab0eda54112d3b0/users/c60c22e5a7b041feb68f7e421c9bf0fd/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.464494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=823fe41ae6e243848ab0eda54112d3b0, user_id=c60c22e5a7b041feb68f7e421c9bf0fd, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.468115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.482287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1341/2688] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:34 2022] PUT /identity/v3/projects/823fe41ae6e243848ab0eda54112d3b0/users/c60c22e5a7b041feb68f7e421c9bf0fd/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:34.492740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c336e-459b-498e-ab26-32070d81f693 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.493259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c336e-459b-498e-ab26-32070d81f693 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.493381 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c336e-459b-498e-ab26-32070d81f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.527100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-298c336e-459b-498e-ab26-32070d81f693 None None] MFA Rules not processed for user `c60c22e5a7b041feb68f7e421c9bf0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:34.528965 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-298c336e-459b-498e-ab26-32070d81f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:34.603796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1348/2689] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:34.615493 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.617469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:34.685392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339b3f4-1ee7-427f-b875-259602ac0adf admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.689441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.690100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.690382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.690663 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.701123 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.702679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.714878 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1342/2690] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:34.796532 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c336e-459b-498e-ab26-32070d81f693 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:34.807380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c336e-459b-498e-ab26-32070d81f693 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:34.814486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37527b8c-280b-4cdc-88c0-38c306db5816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:34.815583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527b8c-280b-4cdc-88c0-38c306db5816 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:34.816020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527b8c-280b-4cdc-88c0-38c306db5816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:34.816441 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527b8c-280b-4cdc-88c0-38c306db5816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:34.902368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37527b8c-280b-4cdc-88c0-38c306db5816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:34.904357 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37527b8c-280b-4cdc-88c0-38c306db5816 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:34.920475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1349/2691] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:35.316654 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:35.336306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:35.433931 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3943d0-69fc-4428-a177-2c03a7a79571 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:35.445919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:35.447051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:35.447405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:35.447740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] PATH_INFO: `/v3/users/9eff1a650e7b4093851d9b44cbcdbe5e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:35.449341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eff1a650e7b4093851d9b44cbcdbe5e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:35.451552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:35.523869 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eff1a650e7b4093851d9b44cbcdbe5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:35.524749 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eff1a650e7b4093851d9b44cbcdbe5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:35.525420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eff1a650e7b4093851d9b44cbcdbe5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:35.536080 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eff1a650e7b4093851d9b44cbcdbe5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:35.552375 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eff1a650e7b4093851d9b44cbcdbe5e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:35.566258 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1343/2692] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:35 2022] DELETE /identity/v3/users/9eff1a650e7b4093851d9b44cbcdbe5e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:35.571064 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:35.571609 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:35.571772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:35.592597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:35.593822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:35.660691 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1350/2693] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:35.670859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:35.672347 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:35.732980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef717105-38cc-4328-942a-fd794a034cfc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:35.737047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:35.737789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:35.738149 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:35.738502 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:35.749820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:35.751378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:35.762341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1344/2694] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:36.066465 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:36.067988 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:36.081617 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda01d8a-0c9a-4704-a43f-5f6e9f74937d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:36.086669 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:36.087641 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:36.087990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:36.088339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] PATH_INFO: `/v3/projects/6e95855d611b4605b3df2a24da1af95b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:36.089593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e95855d611b4605b3df2a24da1af95b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:36.092482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:36.105122 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=100 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:36.145132 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e95855d611b4605b3df2a24da1af95b', 'request_id': 'req-8115fdf0-c794-44c2-a13b-82715682d1c1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.151489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e95855d611b4605b3df2a24da1af95b', 'request_id': 'req-8115fdf0-c794-44c2-a13b-82715682d1c1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.159584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e95855d611b4605b3df2a24da1af95b', 'request_id': 'req-8115fdf0-c794-44c2-a13b-82715682d1c1'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.172414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] The token cache is being invalidate because project 6e95855d611b4605b3df2a24da1af95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:36.172704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.173358 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.175046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1351/2695] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:36 2022] DELETE /identity/v3/projects/6e95855d611b4605b3df2a24da1af95b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:36.180800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:36.194114 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f52f-778b-4f4f-a7b8-0fc35bf4038e service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:36.219864 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:36.258420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:36.259433 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:36.259656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:36.259938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] PATH_INFO: `/v3/users/c60c22e5a7b041feb68f7e421c9bf0fd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:36.261120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60c22e5a7b041feb68f7e421c9bf0fd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:36.263732 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:36.312323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60c22e5a7b041feb68f7e421c9bf0fd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.312994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60c22e5a7b041feb68f7e421c9bf0fd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.313597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60c22e5a7b041feb68f7e421c9bf0fd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.320526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60c22e5a7b041feb68f7e421c9bf0fd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.333683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e2d92ae4-be6c-4efa-8a73-30f8f65dbbdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60c22e5a7b041feb68f7e421c9bf0fd'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.347818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1345/2696] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:36 2022] DELETE /identity/v3/users/c60c22e5a7b041feb68f7e421c9bf0fd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:36.660149 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:36.661686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:36.697512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115fdf0-c794-44c2-a13b-82715682d1c1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:36.761237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:36.761981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:36.762273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:36.763040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] PATH_INFO: `/v3/projects/823fe41ae6e243848ab0eda54112d3b0` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:36.764287 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=823fe41ae6e243848ab0eda54112d3b0)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:36.766261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:36.821903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823fe41ae6e243848ab0eda54112d3b0', 'request_id': 'req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.828673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823fe41ae6e243848ab0eda54112d3b0', 'request_id': 'req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.836712 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823fe41ae6e243848ab0eda54112d3b0', 'request_id': 'req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.849162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] The token cache is being invalidate because project 823fe41ae6e243848ab0eda54112d3b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:36.849451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.850106 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-ad2839d9-c91a-47c5-9f1c-398d69b8afd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:36.851705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1352/2697] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:36 2022] DELETE /identity/v3/projects/823fe41ae6e243848ab0eda54112d3b0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:36.857225 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:36.857700 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:36.857801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:36.877574 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:36.878810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:36.943850 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1346/2698] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:36.953526 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:36.953817 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:36.954104 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:36.981045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:36.981045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:36.991245 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1353/2699] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:36.995648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.006179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052c379-5b89-4715-abca-d493c704bf93 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.011109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.011936 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.012144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.013019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.015194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:37.015194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.015670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.031601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1347/2700] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:15:37.033451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.043370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9d8c-d20b-451b-a458-c7f787ce9232 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.049852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.050619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.050933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.051211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.051911 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.053822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.075315 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:37.083689 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1354/2701] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:37 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:37.087805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.099432 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e758e3-bd1e-4df2-8136-126bdf2dfb6f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.128916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:37.159932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.160614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.160905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.161191 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.161880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.163896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.186497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1348/2702] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:37 2022] POST /identity/v3/users => generated 280 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:37.190774 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.202596 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f836-7bb5-4f6c-a20f-be574b7067f3 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.207588 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.208351 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.208643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.208930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.209660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.211435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.225412 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1355/2703] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:37.229413 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.239594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e38d5-1eb7-4252-abdc-41b3867b224c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.244240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd585ef-84de-4278-af83-7e57ab11221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.244896 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd585ef-84de-4278-af83-7e57ab11221b admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.245185 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd585ef-84de-4278-af83-7e57ab11221b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.245467 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd585ef-84de-4278-af83-7e57ab11221b admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.251098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd585ef-84de-4278-af83-7e57ab11221b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.252751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd585ef-84de-4278-af83-7e57ab11221b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.263951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1349/2704] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:37.268059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.278130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e730d9b-c2a0-4519-a135-11371fbec077 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.283901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.284794 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.285083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.285368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] PATH_INFO: `/v3/projects/bec3d07935944cb3a953811db585150a/users/359a701197a64a669fecef2578536430/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.302053 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec3d07935944cb3a953811db585150a, user_id=359a701197a64a669fecef2578536430, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.304756 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.317917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1356/2705] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:37 2022] PUT /identity/v3/projects/bec3d07935944cb3a953811db585150a/users/359a701197a64a669fecef2578536430/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:37.329471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.329796 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.330079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.358602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] MFA Rules not processed for user `359a701197a64a669fecef2578536430`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:37.360731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:37.439988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1350/2706] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:37.471256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.473305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:37.536704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2668f81-b2d1-4501-a6a6-13089b86f1a6 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.540587 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dffb4-66d3-454b-bb22-75adc7f419cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.541240 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dffb4-66d3-454b-bb22-75adc7f419cc service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.541525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dffb4-66d3-454b-bb22-75adc7f419cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.541822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dffb4-66d3-454b-bb22-75adc7f419cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.552866 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dffb4-66d3-454b-bb22-75adc7f419cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.554332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dffb4-66d3-454b-bb22-75adc7f419cc service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.564765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1357/2707] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:37 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:37.580136 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.652538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4e073-cbf3-4ea5-9a5d-2e0a2c5dde9a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.658826 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.660430 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.660897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.661409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] PATH_INFO: `/v3/users/359a701197a64a669fecef2578536430` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.663096 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=359a701197a64a669fecef2578536430)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.665063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.718967 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:37.726423 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359a701197a64a669fecef2578536430'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:37.727579 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359a701197a64a669fecef2578536430'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:37.728298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359a701197a64a669fecef2578536430'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:37.736659 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359a701197a64a669fecef2578536430'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:37.752798 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359a701197a64a669fecef2578536430'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:37.769795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1351/2708] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:37 2022] DELETE /identity/v3/users/359a701197a64a669fecef2578536430 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:37.774428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.777239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.777239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.793368 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:37.795760 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:37.878345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1358/2709] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:37.889287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:37.890708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:37.953973 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486013c-4679-48de-917f-d66c659747cb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:37.957801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73313a32-50a7-4d2b-a957-be989a4d01ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:37.958490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313a32-50a7-4d2b-a957-be989a4d01ad service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:37.958775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313a32-50a7-4d2b-a957-be989a4d01ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:37.959088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313a32-50a7-4d2b-a957-be989a4d01ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:37.969354 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73313a32-50a7-4d2b-a957-be989a4d01ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:37.970799 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73313a32-50a7-4d2b-a957-be989a4d01ad service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:37.982770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1352/2710] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:38.328572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.330043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:38.351887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d621d36-ecca-40c8-8d6a-fdfbbd6dce41 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.359400 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.360250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.360580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.360833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] PATH_INFO: `/v3/projects/bec3d07935944cb3a953811db585150a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.362583 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec3d07935944cb3a953811db585150a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.365353 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.430705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec3d07935944cb3a953811db585150a', 'request_id': 'req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:38.438637 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec3d07935944cb3a953811db585150a', 'request_id': 'req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:38.447429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec3d07935944cb3a953811db585150a', 'request_id': 'req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:38.451404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:38.463173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] The token cache is being invalidate because project bec3d07935944cb3a953811db585150a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:38.463522 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:38.464401 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-67ed2f82-90f7-4fa3-b3ed-3d216880e57a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:38.466133 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1359/2711] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:38 2022] DELETE /identity/v3/projects/bec3d07935944cb3a953811db585150a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:38.471575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.472025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.472520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.494220 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:38.496251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:38.568088 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1353/2712] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:38.576769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.576993 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.577236 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.596201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:38.597418 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:38.605576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1360/2713] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:38.610107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.619761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f1760-7bc3-4a4a-a9aa-c399d4d3d86c None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.626049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.626734 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.627048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.627328 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.628124 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:38.628450 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.630049 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.642707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1354/2714] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:15:38.647328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.657001 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560056d-b6be-491d-a6a6-3be28ff9bb3e None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.661974 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.662645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.662970 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.663256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.663951 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.665874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.697449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1361/2715] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:38 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:38.698907 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.712824 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506f570-10d7-4c4b-8c9e-47a409d01d56 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.763440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:38.777223 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.777894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.778321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.778638 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.779383 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.781329 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.804272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1355/2716] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:38 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:38.805875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.817077 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85843a-bfc1-4b7a-a6a0-0bcd8d686113 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.821798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.822460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.822750 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.823072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.823826 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.826007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.836626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1362/2717] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:38.840546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.851362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda3053-254a-4720-a21a-1f4974682fa9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.856731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67e278-eda8-43d2-9a80-6a2c50c95720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.857412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67e278-eda8-43d2-9a80-6a2c50c95720 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.857708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67e278-eda8-43d2-9a80-6a2c50c95720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.858015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67e278-eda8-43d2-9a80-6a2c50c95720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.858778 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67e278-eda8-43d2-9a80-6a2c50c95720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.860409 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67e278-eda8-43d2-9a80-6a2c50c95720 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.871284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1356/2718] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:38.877875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:38.890776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899dfdc-f874-441c-860f-c82c0fe91e05 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:38.895757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:38.896696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.896992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.897280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] PATH_INFO: `/v3/projects/336d61b407d845598b4de952651ddb0e/users/3313d97995e745788377afac13f965a7/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:38.912598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336d61b407d845598b4de952651ddb0e, user_id=3313d97995e745788377afac13f965a7, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:38.915346 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:38.929632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1363/2719] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:38 2022] PUT /identity/v3/projects/336d61b407d845598b4de952651ddb0e/users/3313d97995e745788377afac13f965a7/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:38.995647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:38.996230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:38.996230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.026313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] MFA Rules not processed for user `3313d97995e745788377afac13f965a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:39.027567 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:39.106345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1357/2720] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:39.112627 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.177540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af2ce5-8f95-4e3a-a6f1-decf0bd040e1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.182411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.183131 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.183447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.183742 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.184584 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.186652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.216273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1364/2721] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:39.221873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.234216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111860e8-311c-476b-a250-ca6a039d9186 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.297817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.298511 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.298807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.299138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.300816 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.302593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.329919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1358/2722] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:39 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:39.334038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.347058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef38220-c272-41a5-b259-0926f7aa42b0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.351743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.352408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.352694 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.352972 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.353688 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.355395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.367673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1365/2723] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:39.371645 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.382704 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0c84-c16a-4d52-b524-c6a94b895a56 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.388222 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44fbc7-ff19-483c-9cdb-e9ada69c0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.388954 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44fbc7-ff19-483c-9cdb-e9ada69c0ada admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.389249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44fbc7-ff19-483c-9cdb-e9ada69c0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.389594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44fbc7-ff19-483c-9cdb-e9ada69c0ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.390689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44fbc7-ff19-483c-9cdb-e9ada69c0ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.392455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44fbc7-ff19-483c-9cdb-e9ada69c0ada admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.408980 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1359/2724] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:39.410545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.425469 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773823e-72de-4d0f-827f-8b033f0318ae admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.430752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.431673 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.431961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.432239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] PATH_INFO: `/v3/projects/fb311262a5574880bee5e10df6ab7065/users/e9b194f4b82340e5935f0db329928249/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.448427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb311262a5574880bee5e10df6ab7065, user_id=e9b194f4b82340e5935f0db329928249, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.452361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.468404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1366/2725] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:39 2022] PUT /identity/v3/projects/fb311262a5574880bee5e10df6ab7065/users/e9b194f4b82340e5935f0db329928249/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:39.479205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.479587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.479940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.511698 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] MFA Rules not processed for user `e9b194f4b82340e5935f0db329928249`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:39.513440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:39.548495 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:39.605876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1360/2726] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:39.612502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.663142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:39.688279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2042ff-3713-4607-8d37-ec21f5632436 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.693566 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.694307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.694622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.694924 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.695611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.697554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.727471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1367/2727] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:39.734017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.746479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74e67f-3a0b-4997-9c08-037fe328ce88 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.826958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.827649 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.828093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.828589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.829301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.831270 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.855251 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1361/2728] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:39 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:39.859417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.870377 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97584b-ccf0-46c5-8493-a9c0d4eb4f01 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.876561 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.877273 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.877560 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.877839 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.878570 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.880218 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.892901 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1368/2729] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:39.897288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.908405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fb141-d2c6-4be6-9050-94e23d7f093f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:39.913897 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:39.914794 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:39.915106 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:39.915384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] PATH_INFO: `/v3/projects/776d9efcb6e64971a3e8d1ad5b4cfc19/users/27e8ecde51764bfb8f4192fabb75a44c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:39.933434 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=776d9efcb6e64971a3e8d1ad5b4cfc19, user_id=27e8ecde51764bfb8f4192fabb75a44c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:39.936764 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:39.951962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1362/2730] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:39 2022] PUT /identity/v3/projects/776d9efcb6e64971a3e8d1ad5b4cfc19/users/27e8ecde51764bfb8f4192fabb75a44c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:39.956040 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:39.967558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1a23-0e56-4a27-b481-19b66d7725e9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:40.032298 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67faf4-a8e7-4f23-be20-552f8629a9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:40.032983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67faf4-a8e7-4f23-be20-552f8629a9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:40.033271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67faf4-a8e7-4f23-be20-552f8629a9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:40.033555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67faf4-a8e7-4f23-be20-552f8629a9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:40.034304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67faf4-a8e7-4f23-be20-552f8629a9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:40.036002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67faf4-a8e7-4f23-be20-552f8629a9e5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:40.049661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1369/2731] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:40.053854 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:40.068541 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956ce75-df30-4750-be86-8f6bca6c6cfb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:40.073597 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:40.074592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:40.074910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:40.075788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] PATH_INFO: `/v3/projects/776d9efcb6e64971a3e8d1ad5b4cfc19/users/27e8ecde51764bfb8f4192fabb75a44c/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:40.078283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=776d9efcb6e64971a3e8d1ad5b4cfc19, user_id=27e8ecde51764bfb8f4192fabb75a44c, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:40.081004 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:40.093285 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1363/2732] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:40 2022] PUT /identity/v3/projects/776d9efcb6e64971a3e8d1ad5b4cfc19/users/27e8ecde51764bfb8f4192fabb75a44c/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:40.112128 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:40.112423 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:40.112720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:40.144390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] MFA Rules not processed for user `27e8ecde51764bfb8f4192fabb75a44c`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:40.145528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:40.220747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1370/2733] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:40.234273 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:40.236005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:40.306055 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066d181-9a3e-4631-a8a2-a563f6e2b1ca admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:40.310860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:40.311158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:40.311444 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:40.311721 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:40.351837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:40.382093 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:40.383575 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:40.393520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1364/2734] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:42.962695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:42.973153 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f375-3266-4140-be9b-a88c904b7fa4 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:42.976950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:42.977619 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:42.977908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:42.978192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:43.011728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:43.044549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:43.046058 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:43.055344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1371/2735] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:43.761253 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:43.770655 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50967e2-a41f-4741-8add-cbceff6748a8 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:43.775063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc114e-8797-4211-8f1a-5b8f4d9c8d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:43.775722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc114e-8797-4211-8f1a-5b8f4d9c8d37 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:43.776013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc114e-8797-4211-8f1a-5b8f4d9c8d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:43.776293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc114e-8797-4211-8f1a-5b8f4d9c8d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:43.786198 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc114e-8797-4211-8f1a-5b8f4d9c8d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:43.787677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc114e-8797-4211-8f1a-5b8f4d9c8d37 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:43.805881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1365/2736] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:50.001045 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:50.070226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b248-cd77-451c-b4a3-a0b3f159d7f9 service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:50.075554 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:50.076574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.077029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.077378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] PATH_INFO: `/v3/users/3313d97995e745788377afac13f965a7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.078918 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3313d97995e745788377afac13f965a7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:50.081231 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:50.133557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3313d97995e745788377afac13f965a7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.134208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3313d97995e745788377afac13f965a7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.142723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3313d97995e745788377afac13f965a7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.143459 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3313d97995e745788377afac13f965a7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.151507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3313d97995e745788377afac13f965a7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.172017 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1372/2737] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:50 2022] DELETE /identity/v3/users/3313d97995e745788377afac13f965a7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:50.177578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.178257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.178257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.196521 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:50.197758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:50.274823 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1366/2738] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:50.284520 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:50.286033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:50.361565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157748-7710-42cf-ab81-d5211c508d9b admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:50.369792 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a84a9-85b6-46d1-b07b-afaf4ddc4b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:50.370499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a84a9-85b6-46d1-b07b-afaf4ddc4b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.370811 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a84a9-85b6-46d1-b07b-afaf4ddc4b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.371118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a84a9-85b6-46d1-b07b-afaf4ddc4b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.371483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:50.373216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:50.378536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:50.383110 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a84a9-85b6-46d1-b07b-afaf4ddc4b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:50.385600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a84a9-85b6-46d1-b07b-afaf4ddc4b31 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:50.386549 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e08fd3-6ae2-48bf-8868-a1e9c2b411ab None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:50.392138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:50.393059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.393359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.393640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] PATH_INFO: `/v3/users/e2d026e6b68e46fda6091c84ec4ae000` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.395121 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d026e6b68e46fda6091c84ec4ae000)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:50.397119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:50.401120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1373/2739] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:50.473039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d026e6b68e46fda6091c84ec4ae000'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.474318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d026e6b68e46fda6091c84ec4ae000'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.475156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d026e6b68e46fda6091c84ec4ae000'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.488386 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d026e6b68e46fda6091c84ec4ae000'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.506046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d026e6b68e46fda6091c84ec4ae000'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.524289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1367/2740] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:50 2022] DELETE /identity/v3/users/e2d026e6b68e46fda6091c84ec4ae000 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:50.530904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.531507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.531968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.554272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:50.555676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:50.640510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1374/2741] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:50.656791 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:50.658437 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:50.697592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:50.735272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8327dd-9b06-484e-a6be-97eaa6c3e66d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:50.740926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:50.741511 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:50.742147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.742440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.742718 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.743145 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:50.757411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf801589-f288-40e0-be9e-88cf3bfe11aa None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:50.760581 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:50.763384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:50.764802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.764930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:50.765140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.765376 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] PATH_INFO: `/v3/projects/336d61b407d845598b4de952651ddb0e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.767961 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336d61b407d845598b4de952651ddb0e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:50.770034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:50.784469 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1368/2742] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:15:50.832089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336d61b407d845598b4de952651ddb0e', 'request_id': 'req-119b6414-6226-4010-914c-c19535a666c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.839529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336d61b407d845598b4de952651ddb0e', 'request_id': 'req-119b6414-6226-4010-914c-c19535a666c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.849211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336d61b407d845598b4de952651ddb0e', 'request_id': 'req-119b6414-6226-4010-914c-c19535a666c7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.862735 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] The token cache is being invalidate because project 336d61b407d845598b4de952651ddb0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:50.863083 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.863754 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:50.865497 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1375/2743] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:50 2022] DELETE /identity/v3/projects/336d61b407d845598b4de952651ddb0e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:50.869682 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:50.881356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44466ac0-4033-42b5-9fcf-c03d5c796320 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:50.959746 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:50.960805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:50.961165 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:50.961516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] PATH_INFO: `/v3/users/e9b194f4b82340e5935f0db329928249` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:50.963143 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b194f4b82340e5935f0db329928249)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:50.965504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:51.027154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b194f4b82340e5935f0db329928249'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.027981 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b194f4b82340e5935f0db329928249'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.029670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b194f4b82340e5935f0db329928249'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.040841 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b194f4b82340e5935f0db329928249'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.057024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b194f4b82340e5935f0db329928249'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.077565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1369/2744] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:50 2022] DELETE /identity/v3/users/e9b194f4b82340e5935f0db329928249 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:51.119047 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:51.120509 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:51.137174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b6414-6226-4010-914c-c19535a666c7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:51.225003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:51.225929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:51.226219 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:51.226501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] PATH_INFO: `/v3/projects/26bf66c7d87b4aa8a95c10c4cd549f64` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:51.227819 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bf66c7d87b4aa8a95c10c4cd549f64)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:51.229814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:51.258202 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:51.295485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bf66c7d87b4aa8a95c10c4cd549f64', 'request_id': 'req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.302707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bf66c7d87b4aa8a95c10c4cd549f64', 'request_id': 'req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.312033 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bf66c7d87b4aa8a95c10c4cd549f64', 'request_id': 'req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.329580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] The token cache is being invalidate because project 26bf66c7d87b4aa8a95c10c4cd549f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:51.329944 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.330707 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.332551 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1376/2745] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:51 2022] DELETE /identity/v3/projects/26bf66c7d87b4aa8a95c10c4cd549f64 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:51.336942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:51.338731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:51.355953 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e066e0-3d61-4c3a-b1e3-81c4f4c5c251 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:51.421457 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:51.426271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:51.428027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:51.433374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:51.439021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:51.439021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:51.439021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] PATH_INFO: `/v3/users/dd3c6a1615e64a82afd819707f0d123d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:51.439021 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3c6a1615e64a82afd819707f0d123d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:51.440210 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:51.445789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213ea8d-2ec1-436f-95c3-9a5c1f4ca02f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:51.464307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:51.465632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:51.465632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:51.465868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] PATH_INFO: `/v3/projects/fb311262a5574880bee5e10df6ab7065` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:51.467130 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb311262a5574880bee5e10df6ab7065)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:51.469884 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:51.511178 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3c6a1615e64a82afd819707f0d123d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.513374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3c6a1615e64a82afd819707f0d123d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.516565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3c6a1615e64a82afd819707f0d123d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.525471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3c6a1615e64a82afd819707f0d123d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.541079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3c6a1615e64a82afd819707f0d123d'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.547802 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb311262a5574880bee5e10df6ab7065', 'request_id': 'req-9f462e03-39a4-4940-afe0-a25be5009e90'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.555658 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb311262a5574880bee5e10df6ab7065', 'request_id': 'req-9f462e03-39a4-4940-afe0-a25be5009e90'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.563907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb311262a5574880bee5e10df6ab7065', 'request_id': 'req-9f462e03-39a4-4940-afe0-a25be5009e90'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.567740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1370/2746] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:51 2022] DELETE /identity/v3/users/dd3c6a1615e64a82afd819707f0d123d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:51.577404 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] The token cache is being invalidate because project fb311262a5574880bee5e10df6ab7065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:51.577743 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.578573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.580737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1377/2747] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:51 2022] DELETE /identity/v3/projects/fb311262a5574880bee5e10df6ab7065 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:51.585003 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:51.586587 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:51.668390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3bc14-bb1a-452b-832d-646b23f16dce admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:51.674179 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:51.675103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:51.675697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:51.675820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] PATH_INFO: `/v3/users/27e8ecde51764bfb8f4192fabb75a44c` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:51.677378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e8ecde51764bfb8f4192fabb75a44c)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:51.679573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:51.756719 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e8ecde51764bfb8f4192fabb75a44c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.757550 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e8ecde51764bfb8f4192fabb75a44c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.758322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e8ecde51764bfb8f4192fabb75a44c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.767894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e8ecde51764bfb8f4192fabb75a44c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.782286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e8ecde51764bfb8f4192fabb75a44c'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:51.784277 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:51.799818 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1371/2748] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:15:51 2022] DELETE /identity/v3/users/27e8ecde51764bfb8f4192fabb75a44c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:51.894773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:51.897573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.003549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f462e03-39a4-4940-afe0-a25be5009e90 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.048591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:52.084319 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.086199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.086630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.086976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] PATH_INFO: `/v3/projects/40fdb72856d44b9cb4d71353fe00a843` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.088813 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fdb72856d44b9cb4d71353fe00a843)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.091616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.128992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.131283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.155129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d06f1-1509-4d7b-9f17-85727edb90a8 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.162223 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fdb72856d44b9cb4d71353fe00a843', 'request_id': 'req-907c5dad-c635-43e9-bd23-65db3034bfe4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.172417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fdb72856d44b9cb4d71353fe00a843', 'request_id': 'req-907c5dad-c635-43e9-bd23-65db3034bfe4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.193684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fdb72856d44b9cb4d71353fe00a843', 'request_id': 'req-907c5dad-c635-43e9-bd23-65db3034bfe4'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.207572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] The token cache is being invalidate because project 40fdb72856d44b9cb4d71353fe00a843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:52.207997 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.208657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-907c5dad-c635-43e9-bd23-65db3034bfe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.210492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1378/2749] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:51 2022] DELETE /identity/v3/projects/40fdb72856d44b9cb4d71353fe00a843 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:52.214930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11117d-474e-45c6-9c6a-c70d60a7e0e2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.215217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11117d-474e-45c6-9c6a-c70d60a7e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.215558 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11117d-474e-45c6-9c6a-c70d60a7e0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.229385 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.230261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.230553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.230860 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] PATH_INFO: `/v3/projects/776d9efcb6e64971a3e8d1ad5b4cfc19` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.232909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=776d9efcb6e64971a3e8d1ad5b4cfc19)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.234899 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.240007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8b11117d-474e-45c6-9c6a-c70d60a7e0e2 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:52.241310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8b11117d-474e-45c6-9c6a-c70d60a7e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.251173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1379/2750] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:52.258998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.259310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.259597 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.279908 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:52.281455 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.295131 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776d9efcb6e64971a3e8d1ad5b4cfc19', 'request_id': 'req-7d90668b-d7d9-416b-ac47-209380e1a61f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.302438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776d9efcb6e64971a3e8d1ad5b4cfc19', 'request_id': 'req-7d90668b-d7d9-416b-ac47-209380e1a61f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.310889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776d9efcb6e64971a3e8d1ad5b4cfc19', 'request_id': 'req-7d90668b-d7d9-416b-ac47-209380e1a61f'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.327001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] The token cache is being invalidate because project 776d9efcb6e64971a3e8d1ad5b4cfc19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:15:52.327283 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.327963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-7d90668b-d7d9-416b-ac47-209380e1a61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:15:52.329547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1372/2751] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:15:52 2022] DELETE /identity/v3/projects/776d9efcb6e64971a3e8d1ad5b4cfc19 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:52.334367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722895d7-7982-4dd4-a978-4a4d99a61fc5 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.334661 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722895d7-7982-4dd4-a978-4a4d99a61fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.334992 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722895d7-7982-4dd4-a978-4a4d99a61fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.355235 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-722895d7-7982-4dd4-a978-4a4d99a61fc5 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:52.357009 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-722895d7-7982-4dd4-a978-4a4d99a61fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.364278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1380/2752] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:52.364994 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1373/2753] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:52.370014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.373305 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.373792 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.374082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.382698 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7da2a-6862-427f-a5e3-307f70f32281 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.388344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.389026 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.389316 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.389600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.390331 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:52.390647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.392341 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.394735 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:52.395874 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.407838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1374/2755] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:52.408005 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1381/2755] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:15:52.412157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.412767 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.423024 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ba56-54fa-4f8c-bcf8-199f5920fc04 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.423428 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a628b-57ab-40d7-a3fa-0d977012bdc0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.427632 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.428282 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.428565 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.428843 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.428998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.429586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:15:52.429775 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.429879 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.430116 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.430406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.431155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.431636 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.433062 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.445506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1375/2756] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:15:52.450078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.461078 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7efc08-21fb-4409-9fbf-80d366d33781 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.465048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1382/2757] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:52.465934 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.466592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.466908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.467189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.467882 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.469435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.469790 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.504743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1376/2758] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/projects => generated 397 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:52.509590 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.510618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:52.548233 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63183039-a778-4e3b-af01-42f295f40b53 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.550536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:52.593316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e176e-f85c-43a0-8791-de697ab09249 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.599348 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.600120 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.600504 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.600867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.601643 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.603807 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.628379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.629270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.629620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.629995 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.630753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.631527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1377/2759] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:52.634495 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.637229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.648853 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d891123-9725-4254-b1cf-b761b60ccd6c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.654092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.654788 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.655119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.655402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.656130 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.657733 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.658188 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1383/2760] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:52.662553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.668402 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1378/2761] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:52.672175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.675197 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522b403-180d-4f15-8550-fdc4e64e31f0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.679787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.680451 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.680736 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.681014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.681762 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c844f06-b098-4fc1-9756-825719a36d30 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.681883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.683467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.687292 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.688030 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.688388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.688743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.689543 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.691238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.695184 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1384/2762] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:52.699166 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.703431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1379/2763] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:52.708047 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.709916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf41809-1fdf-44af-a8cf-c764ed78d43c admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.714618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.715303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.715633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.716003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.717311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.718743 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b556c0-66ef-4596-aba3-833cc831c254 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.718954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.723621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47d45-ba4a-48d7-9b5e-5b8d0144e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.724490 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47d45-ba4a-48d7-9b5e-5b8d0144e3de admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.724773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47d45-ba4a-48d7-9b5e-5b8d0144e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.725051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47d45-ba4a-48d7-9b5e-5b8d0144e3de admin admin] PATH_INFO: `/v3/projects/8adc9dc2ac264f1ab78d82569657048c/users/3da8a12486be4066809338489e626feb/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.731705 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1385/2764] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:52.735976 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.742736 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c47d45-ba4a-48d7-9b5e-5b8d0144e3de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8adc9dc2ac264f1ab78d82569657048c, user_id=3da8a12486be4066809338489e626feb, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.745442 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c47d45-ba4a-48d7-9b5e-5b8d0144e3de admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.745981 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8024bb0-d476-47de-8c32-fa98e6d389ab admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.751835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d908-baec-47e1-8c5f-d57e9e4c8177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.752766 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d908-baec-47e1-8c5f-d57e9e4c8177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.753052 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d908-baec-47e1-8c5f-d57e9e4c8177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.753384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d908-baec-47e1-8c5f-d57e9e4c8177 admin admin] PATH_INFO: `/v3/projects/0e45c91aa05d4ff6855e9337c068874a/users/1ab463bcd0a648ccba5dfe1b22369905/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.762129 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1380/2765] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:52 2022] PUT /identity/v3/projects/8adc9dc2ac264f1ab78d82569657048c/users/3da8a12486be4066809338489e626feb/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:52.770553 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683d908-baec-47e1-8c5f-d57e9e4c8177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e45c91aa05d4ff6855e9337c068874a, user_id=1ab463bcd0a648ccba5dfe1b22369905, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.773607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683d908-baec-47e1-8c5f-d57e9e4c8177 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.773977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.774324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.774634 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.787632 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1386/2766] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:52 2022] PUT /identity/v3/projects/0e45c91aa05d4ff6855e9337c068874a/users/1ab463bcd0a648ccba5dfe1b22369905/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:15:52.799339 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.799626 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.800160 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.804422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] MFA Rules not processed for user `3da8a12486be4066809338489e626feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:52.805857 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.831082 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] MFA Rules not processed for user `1ab463bcd0a648ccba5dfe1b22369905`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:52.832535 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.884189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1381/2767] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:52.897784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.899241 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:52.917770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1387/2768] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:15:52.923074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:52.934634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:52.963785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557feb64-280b-4cb2-bca8-41a8ff2cb78a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.969774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.970566 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.970876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.971230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:52.982479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:52.983966 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:52.992620 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335f27e-0eca-4386-9a6d-985d8802af3b None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:52.993592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1382/2769] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:15:52.998349 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:52.999042 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:52.999330 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:52.999599 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.000289 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.002229 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.038457 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1388/2770] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:15:52 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:15:53.042793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:53.055946 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c9a1-ae40-487e-af2d-1fbfeba088b2 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:53.067984 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:15:53.140039 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:53.140959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.141429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.141859 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.143204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.147440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.174959 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1383/2771] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:15:53 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:15:53.180541 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:53.192910 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298339b-71cc-4e1d-a030-4018e8b4abff admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:53.197716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:53.198384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.198670 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.198985 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.199709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.202968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.214435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1389/2772] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:15:53.219837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:53.230982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b830ed-7c15-4554-bd65-8141c20d2dd7 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:53.236978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:53.237982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.238274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.238601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] PATH_INFO: `/v3/projects/a1d59bb9d36a41639b70f804cd0c35f2/users/ce99ddfdd0124c43af3d26c0a59d04e7/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.255809 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d59bb9d36a41639b70f804cd0c35f2, user_id=ce99ddfdd0124c43af3d26c0a59d04e7, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.259017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.274556 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1384/2773] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:53 2022] PUT /identity/v3/projects/a1d59bb9d36a41639b70f804cd0c35f2/users/ce99ddfdd0124c43af3d26c0a59d04e7/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:53.279141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:53.292214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667313a-09a5-4ac8-869f-497478e37188 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:53.360549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908841b-07ae-499c-881c-d0b8e988fb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:53.361234 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908841b-07ae-499c-881c-d0b8e988fb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.361544 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908841b-07ae-499c-881c-d0b8e988fb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.361807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908841b-07ae-499c-881c-d0b8e988fb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.363208 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908841b-07ae-499c-881c-d0b8e988fb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.365100 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908841b-07ae-499c-881c-d0b8e988fb0b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.380155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1390/2774] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:15:53.384923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:53.399227 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ddb9c-9ab2-458c-84e0-b7f9a5bad2f5 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:53.404923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:53.405887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.406238 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.406584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] PATH_INFO: `/v3/projects/a1d59bb9d36a41639b70f804cd0c35f2/users/ce99ddfdd0124c43af3d26c0a59d04e7/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.408793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d59bb9d36a41639b70f804cd0c35f2, user_id=ce99ddfdd0124c43af3d26c0a59d04e7, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.411578 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.425692 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1385/2775] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:15:53 2022] PUT /identity/v3/projects/a1d59bb9d36a41639b70f804cd0c35f2/users/ce99ddfdd0124c43af3d26c0a59d04e7/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:15:53.436314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54597a10-10f3-47df-8781-59a789def89c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.436633 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54597a10-10f3-47df-8781-59a789def89c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.436897 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54597a10-10f3-47df-8781-59a789def89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.473111 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-54597a10-10f3-47df-8781-59a789def89c None None] MFA Rules not processed for user `ce99ddfdd0124c43af3d26c0a59d04e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:15:53.474327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-54597a10-10f3-47df-8781-59a789def89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:53.554508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1391/2776] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:15:53.566880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:15:53.568647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:15:53.631025 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478dd7f-8872-4064-a3d8-9d92ae058ad2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:15:53.634793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:15:53.635478 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:15:53.635763 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:15:53.636060 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:15:53.700142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:15:53.703534 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:15:53.712916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1386/2777] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:15:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:03.551806 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:03.708751 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:05.734036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54597a10-10f3-47df-8781-59a789def89c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:05.756746 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54597a10-10f3-47df-8781-59a789def89c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:05.760449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161b750-53c2-488d-a4a4-f5ff867ac73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:05.761098 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161b750-53c2-488d-a4a4-f5ff867ac73a service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:05.761380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161b750-53c2-488d-a4a4-f5ff867ac73a service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:05.761676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161b750-53c2-488d-a4a4-f5ff867ac73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:05.794071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161b750-53c2-488d-a4a4-f5ff867ac73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:05.795580 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161b750-53c2-488d-a4a4-f5ff867ac73a service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:05.810321 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1392/2778] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:11.553377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:11.698485 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1a25-2815-450d-90be-f6b83c0ea3a7 service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:11.698977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:11.700154 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:11.700559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:11.700943 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] PATH_INFO: `/v3/users/3da8a12486be4066809338489e626feb` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:11.702230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3da8a12486be4066809338489e626feb)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:11.704216 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:11.755835 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3da8a12486be4066809338489e626feb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:11.756933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3da8a12486be4066809338489e626feb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:11.757557 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3da8a12486be4066809338489e626feb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:11.764902 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3da8a12486be4066809338489e626feb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:11.777948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3da8a12486be4066809338489e626feb'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:11.794420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1387/2779] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:11 2022] DELETE /identity/v3/users/3da8a12486be4066809338489e626feb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:11.798921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:11.799402 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:11.799523 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:11.850744 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:11.852006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:11.915499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1393/2780] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:16:11.925381 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:11.927074 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:11.978182 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:12.021963 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f94f8-a038-4fa0-aa9d-5d42c23a5a5c admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:12.025667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:12.026319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:12.026605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:12.026993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:12.040909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:12.042520 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:12.055739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1388/2781] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:12.365838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:12.367502 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:12.382015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18d47-e7bc-41c4-9d96-3858e8211f15 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:12.386378 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:12.387815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:12.388112 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:12.388403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] PATH_INFO: `/v3/projects/8adc9dc2ac264f1ab78d82569657048c` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:12.389703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8adc9dc2ac264f1ab78d82569657048c)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:12.391655 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:12.444998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8adc9dc2ac264f1ab78d82569657048c', 'request_id': 'req-4e811b3e-fb6a-494e-b76a-289f307b1d9b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:12.451505 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8adc9dc2ac264f1ab78d82569657048c', 'request_id': 'req-4e811b3e-fb6a-494e-b76a-289f307b1d9b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:12.463890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8adc9dc2ac264f1ab78d82569657048c', 'request_id': 'req-4e811b3e-fb6a-494e-b76a-289f307b1d9b'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:12.475488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] The token cache is being invalidate because project 8adc9dc2ac264f1ab78d82569657048c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:12.475776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:12.476430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-4e811b3e-fb6a-494e-b76a-289f307b1d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:12.477989 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1394/2782] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:12 2022] DELETE /identity/v3/projects/8adc9dc2ac264f1ab78d82569657048c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:22.051044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=19 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:22.472431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:47.313264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:47.314750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:47.328771 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04a234-92c6-404e-8726-37dbf28e9b69 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:47.393648 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:47.394528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:47.394817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:47.395127 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] PATH_INFO: `/v3/users/1ab463bcd0a648ccba5dfe1b22369905` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:47.396318 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab463bcd0a648ccba5dfe1b22369905)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:47.398881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:47.449671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab463bcd0a648ccba5dfe1b22369905'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:47.450440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab463bcd0a648ccba5dfe1b22369905'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:47.451084 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab463bcd0a648ccba5dfe1b22369905'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:47.459172 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab463bcd0a648ccba5dfe1b22369905'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:47.471913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab463bcd0a648ccba5dfe1b22369905'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:47.486170 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1389/2783] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:47 2022] DELETE /identity/v3/users/1ab463bcd0a648ccba5dfe1b22369905 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:16:47.490660 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:47.491203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:47.491314 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:47.511415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:47.512618 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:47.572280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1395/2784] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:47.582205 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:47.583677 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:47.645553 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba7b4-70cc-42b1-ab8b-9ae34ee3f4f1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:47.649362 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:47.650020 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:47.650313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:47.650605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:47.660164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:47.661612 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:47.671460 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1390/2785] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:47.962216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:47.963722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:47.979171 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96324ec-ed05-4db3-b1a5-77a13fc1148a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:47.984304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:47.985242 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:47.985485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:47.985834 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] PATH_INFO: `/v3/projects/0e45c91aa05d4ff6855e9337c068874a` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:47.987055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e45c91aa05d4ff6855e9337c068874a)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:47.989159 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:48.056444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e45c91aa05d4ff6855e9337c068874a', 'request_id': 'req-13795a0c-b860-4ebb-bf75-d6155659d530'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.062977 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e45c91aa05d4ff6855e9337c068874a', 'request_id': 'req-13795a0c-b860-4ebb-bf75-d6155659d530'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.072593 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e45c91aa05d4ff6855e9337c068874a', 'request_id': 'req-13795a0c-b860-4ebb-bf75-d6155659d530'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.085585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] The token cache is being invalidate because project 0e45c91aa05d4ff6855e9337c068874a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:48.085879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.086536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.088118 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1396/2786] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:47 2022] DELETE /identity/v3/projects/0e45c91aa05d4ff6855e9337c068874a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:48.092200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:48.103219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ed58f-8461-4496-a315-ad184fa7857e service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:48.124112 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:48.170724 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:48.171647 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:48.171942 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:48.172229 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] PATH_INFO: `/v3/users/ce99ddfdd0124c43af3d26c0a59d04e7` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:48.173440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce99ddfdd0124c43af3d26c0a59d04e7)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:48.175439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:48.223212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce99ddfdd0124c43af3d26c0a59d04e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.224166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce99ddfdd0124c43af3d26c0a59d04e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.224785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce99ddfdd0124c43af3d26c0a59d04e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.233048 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce99ddfdd0124c43af3d26c0a59d04e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.246982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-b970b343-e0a0-409c-a6f2-b79fba5478af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce99ddfdd0124c43af3d26c0a59d04e7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.265200 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1391/2787] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:48 2022] DELETE /identity/v3/users/ce99ddfdd0124c43af3d26c0a59d04e7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:48.572325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:48.573863 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:48.588224 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13795a0c-b860-4ebb-bf75-d6155659d530 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:48.649210 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:48.650120 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:48.650407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:48.650686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] PATH_INFO: `/v3/projects/a1d59bb9d36a41639b70f804cd0c35f2` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:48.652021 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d59bb9d36a41639b70f804cd0c35f2)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:48.653991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:48.689892 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:48.708770 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d59bb9d36a41639b70f804cd0c35f2', 'request_id': 'req-c9141b72-6476-450e-bc07-05cd95999036'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.717055 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d59bb9d36a41639b70f804cd0c35f2', 'request_id': 'req-c9141b72-6476-450e-bc07-05cd95999036'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.726291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d59bb9d36a41639b70f804cd0c35f2', 'request_id': 'req-c9141b72-6476-450e-bc07-05cd95999036'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.738656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] The token cache is being invalidate because project a1d59bb9d36a41639b70f804cd0c35f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:48.738980 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.739648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c9141b72-6476-450e-bc07-05cd95999036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:48.741196 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1397/2788] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:48 2022] DELETE /identity/v3/projects/a1d59bb9d36a41639b70f804cd0c35f2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:48.746269 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:48.746933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:48.746933 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:48.801605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:48.802950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:48.866916 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1392/2789] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:48.874464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:48.874765 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:48.875081 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:48.895283 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:48.896501 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:48.905327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1398/2790] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:48.909542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:48.919411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fdfdf-efb5-4c3f-8fea-43fc95b8397d None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:48.923876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:48.924522 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:48.924881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:48.925160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:48.925906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:16:48.926212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:48.927849 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:48.941315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1393/2791] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:16:48.946343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:48.955835 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b6d55-50bb-4cde-a017-8729c55a8361 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:48.960653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:48.961411 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:48.961702 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:48.961984 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:48.962675 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:48.964768 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:48.993036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1399/2792] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:16:48 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:16:48.996331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.003321 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:49.008712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bfa47-0f55-4a40-8289-ae8f5f5c8816 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.074773 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.075950 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.076456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.076949 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.078324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.084530 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.119673 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1394/2793] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:16:48 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:16:49.120764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.131726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c136a5-bc42-4754-a910-e53f51d05796 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.137715 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.138549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.138832 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.139270 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.140041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.141719 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.152871 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1400/2794] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:16:49.156891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.167051 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c327ad-077b-417f-b993-47e80e16fd82 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.171667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c92e26-c2a3-4c8e-9764-766c7c0be9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.172320 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c92e26-c2a3-4c8e-9764-766c7c0be9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.172605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c92e26-c2a3-4c8e-9764-766c7c0be9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.172883 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c92e26-c2a3-4c8e-9764-766c7c0be9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.173601 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c92e26-c2a3-4c8e-9764-766c7c0be9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.175215 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c92e26-c2a3-4c8e-9764-766c7c0be9c5 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.185805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1395/2795] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:49.190549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.202211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15607b-3614-47eb-a465-a922200b14f4 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.207261 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.208014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.208282 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.208567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] PATH_INFO: `/v3/projects/d6188fe0c4cd4c3fa342d6a744977bf1/users/628fc1ad42b9491d868ae89129bebb05/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.224029 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6188fe0c4cd4c3fa342d6a744977bf1, user_id=628fc1ad42b9491d868ae89129bebb05, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.226783 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.240791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1401/2796] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:49 2022] PUT /identity/v3/projects/d6188fe0c4cd4c3fa342d6a744977bf1/users/628fc1ad42b9491d868ae89129bebb05/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:49.251591 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.251886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.252166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.280107 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] MFA Rules not processed for user `628fc1ad42b9491d868ae89129bebb05`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:49.281307 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:49.361819 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1396/2797] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:49.366025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.447772 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2ab2f-51b3-47bd-bb90-b5d0b996c3e0 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.453729 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.454407 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.454693 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.455214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.455925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.458044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.488903 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1402/2798] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:16:49 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:16:49.490139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.502944 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d568f-6410-4148-874a-81274af521e0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.559983 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.560685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.560976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.561257 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.561948 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.563900 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.584019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1397/2799] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:16:49 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:16:49.588925 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.599740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d62df-28b4-408b-a891-ce0a42d1b657 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.604916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.605576 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.605865 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.606151 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.606893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.608507 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.620991 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1403/2800] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:49.625287 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.630594 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:49.636960 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b63bb3-22b7-45cc-93f5-9e4a03a4ab3f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.641880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.642761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.643077 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.643355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] PATH_INFO: `/v3/projects/7cbe4c27d0d24f3e8a4ec340d1cbe4ab/users/4f0fb173112b4b5caea72982276cd998/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.658754 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbe4c27d0d24f3e8a4ec340d1cbe4ab, user_id=4f0fb173112b4b5caea72982276cd998, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.661429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.675881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1398/2801] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:49 2022] PUT /identity/v3/projects/7cbe4c27d0d24f3e8a4ec340d1cbe4ab/users/4f0fb173112b4b5caea72982276cd998/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:49.680013 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.690215 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fea69-1e7e-49e5-aa24-4aa4c01a4272 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.711195 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:49.765958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59846a95-9625-4dab-88fd-613429fe0e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.766638 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59846a95-9625-4dab-88fd-613429fe0e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.766954 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59846a95-9625-4dab-88fd-613429fe0e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.767237 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59846a95-9625-4dab-88fd-613429fe0e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.767963 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59846a95-9625-4dab-88fd-613429fe0e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.769581 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59846a95-9625-4dab-88fd-613429fe0e3a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.780572 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1404/2802] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:16:49.784870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.796054 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6d9cd-3854-48be-a853-ff99e41cefbc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:49.800678 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:49.801583 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.801870 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.802148 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] PATH_INFO: `/v3/projects/7cbe4c27d0d24f3e8a4ec340d1cbe4ab/users/4f0fb173112b4b5caea72982276cd998/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.804207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbe4c27d0d24f3e8a4ec340d1cbe4ab, user_id=4f0fb173112b4b5caea72982276cd998, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:49.806923 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:49.816940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1399/2803] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:49 2022] PUT /identity/v3/projects/7cbe4c27d0d24f3e8a4ec340d1cbe4ab/users/4f0fb173112b4b5caea72982276cd998/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:49.827929 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:49.828225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:49.828510 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:49.859194 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] MFA Rules not processed for user `4f0fb173112b4b5caea72982276cd998`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:49.860488 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:49.932023 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1405/2804] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:49.942515 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:49.944243 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:50.002137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3d513-b85e-4155-9beb-b47ae1b14e5f admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:50.006431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa11fc-8a17-45b6-8743-aff97a889786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:50.007289 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa11fc-8a17-45b6-8743-aff97a889786 service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:50.007723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa11fc-8a17-45b6-8743-aff97a889786 service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:50.008022 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa11fc-8a17-45b6-8743-aff97a889786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:50.018063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa11fc-8a17-45b6-8743-aff97a889786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:50.019603 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa11fc-8a17-45b6-8743-aff97a889786 service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:50.030019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1400/2805] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:50.628002 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:50.700437 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd810-5d9b-4005-aaca-9cc2d58b8047 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:50.712477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:50.715252 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:50.715789 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:50.717333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] PATH_INFO: `/v3/users/628fc1ad42b9491d868ae89129bebb05` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:50.718992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628fc1ad42b9491d868ae89129bebb05)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:50.721293 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:50.769791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628fc1ad42b9491d868ae89129bebb05'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:50.770445 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628fc1ad42b9491d868ae89129bebb05'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:50.771721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:50.779713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628fc1ad42b9491d868ae89129bebb05'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:50.780375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628fc1ad42b9491d868ae89129bebb05'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:50.787383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628fc1ad42b9491d868ae89129bebb05'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:50.805753 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1406/2806] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:50 2022] DELETE /identity/v3/users/628fc1ad42b9491d868ae89129bebb05 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:50.810280 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:50.810742 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:50.810895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:50.829059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:50.830264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:50.891964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1401/2807] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:50.902179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:50.903679 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:50.964296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbfa1-c7b2-41d2-9145-22ee315321f9 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:50.967940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:50.968589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:50.968879 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:50.969163 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:50.979458 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:50.980921 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:50.990492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1407/2808] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:16:51.271656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:51.273466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:51.288748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b257db4-6684-4a6f-977c-c57582067fed None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:51.294770 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:51.295748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:51.296036 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:51.296311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] PATH_INFO: `/v3/projects/d6188fe0c4cd4c3fa342d6a744977bf1` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:51.298164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6188fe0c4cd4c3fa342d6a744977bf1)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:51.300125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:51.348403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6188fe0c4cd4c3fa342d6a744977bf1', 'request_id': 'req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.355401 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6188fe0c4cd4c3fa342d6a744977bf1', 'request_id': 'req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.361331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6188fe0c4cd4c3fa342d6a744977bf1', 'request_id': 'req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.363411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:51.376115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] The token cache is being invalidate because project d6188fe0c4cd4c3fa342d6a744977bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:51.376405 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.377052 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.378609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1402/2809] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:51 2022] DELETE /identity/v3/projects/d6188fe0c4cd4c3fa342d6a744977bf1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:51.382525 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:51.392623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447803d-4151-47c6-a341-206af20ccd5f service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:51.450579 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:51.451481 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:51.451769 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:51.452048 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] PATH_INFO: `/v3/users/4f0fb173112b4b5caea72982276cd998` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:51.453235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0fb173112b4b5caea72982276cd998)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:51.455183 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:51.502162 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0fb173112b4b5caea72982276cd998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.502732 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0fb173112b4b5caea72982276cd998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.510499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0fb173112b4b5caea72982276cd998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.511157 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0fb173112b4b5caea72982276cd998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.518812 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-7df42fae-2998-4368-a4ad-6a3ef3da86f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0fb173112b4b5caea72982276cd998'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:51.523622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:51.538246 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1408/2810] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:51 2022] DELETE /identity/v3/users/4f0fb173112b4b5caea72982276cd998 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:51.847726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:51.849519 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:51.873755 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05ef4d-dc2c-4647-9fad-6be2e7d53296 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:51.942608 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:51.943503 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:51.943795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:51.944094 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] PATH_INFO: `/v3/projects/7cbe4c27d0d24f3e8a4ec340d1cbe4ab` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:51.945281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbe4c27d0d24f3e8a4ec340d1cbe4ab)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:51.947249 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:51.994941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbe4c27d0d24f3e8a4ec340d1cbe4ab', 'request_id': 'req-89967e92-4926-4e8c-b769-17ae30fcea82'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:52.000398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbe4c27d0d24f3e8a4ec340d1cbe4ab', 'request_id': 'req-89967e92-4926-4e8c-b769-17ae30fcea82'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:52.006932 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbe4c27d0d24f3e8a4ec340d1cbe4ab', 'request_id': 'req-89967e92-4926-4e8c-b769-17ae30fcea82'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:52.019714 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] The token cache is being invalidate because project 7cbe4c27d0d24f3e8a4ec340d1cbe4ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:52.020137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:52.020761 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-89967e92-4926-4e8c-b769-17ae30fcea82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:52.022286 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1403/2811] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:51 2022] DELETE /identity/v3/projects/7cbe4c27d0d24f3e8a4ec340d1cbe4ab => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:16:52.027943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.028379 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.028518 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.047591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:52.048807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:52.109666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1409/2812] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:52.117419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.117715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.118653 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.139211 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:52.140418 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:52.148456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1404/2813] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:52.152894 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.162614 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d2128-0284-43ee-89c6-0dcdee0466e2 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.167361 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.168041 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.168325 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.168604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.169322 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:16:52.169630 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.171297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.183654 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1410/2814] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:16:52.187926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.198083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e250ed-3011-4745-93f8-25faf88c5d05 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.203155 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.203952 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.204240 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.204774 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.205494 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.207392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.236345 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1405/2815] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:16:52.240492 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.251148 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870997-f6e0-4afe-8b1c-0c51e83ad6b1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.307516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.308201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.308491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.308771 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.309485 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.311521 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.330657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1411/2816] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:16:52.334708 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.344989 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb282b8-bc29-46b7-800b-d622634fa759 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.349978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.350640 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.350976 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.351246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.351970 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.354586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.365750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1406/2817] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:52.370757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.380803 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6f59f-e976-4338-b82a-4881858fd5c1 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.385442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29e249-cc37-4cba-b005-7ab91828583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.386101 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29e249-cc37-4cba-b005-7ab91828583e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.386394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29e249-cc37-4cba-b005-7ab91828583e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.386672 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29e249-cc37-4cba-b005-7ab91828583e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.387431 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29e249-cc37-4cba-b005-7ab91828583e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.389094 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29e249-cc37-4cba-b005-7ab91828583e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.399443 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1412/2818] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:16:52.403901 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.413940 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2c9a5-4eb8-4fe5-96d9-536a37c03817 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.419452 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.420322 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.420614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.420887 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] PATH_INFO: `/v3/projects/f0632bd8beff43ea817a748a09d6f769/users/e2686ee37d334f01b8ddd8cefe09b7b0/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.437260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0632bd8beff43ea817a748a09d6f769, user_id=e2686ee37d334f01b8ddd8cefe09b7b0, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.439951 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.453221 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1407/2819] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:52 2022] PUT /identity/v3/projects/f0632bd8beff43ea817a748a09d6f769/users/e2686ee37d334f01b8ddd8cefe09b7b0/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:16:52.463747 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.464046 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.464333 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.492444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] MFA Rules not processed for user `e2686ee37d334f01b8ddd8cefe09b7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:52.494512 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:52.536278 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:52.565088 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1413/2820] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:52.569487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.588795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:52.635508 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5d032-7508-4f39-bc3f-818dea01085e admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.640167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.640822 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.641109 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.641392 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.642081 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.644043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.671070 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1408/2821] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:16:52.674690 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.685718 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abfe24-527e-4fbe-a3fc-3203975e4117 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.745337 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.745996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.746280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.746562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.747290 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.749386 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.769574 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1414/2822] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:16:52 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:16:52.773562 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.784972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500cd5-fbde-4691-a644-d15d2896a767 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.789564 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.790958 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.791310 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.791600 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.792479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.795331 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.807115 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1409/2823] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:52.812003 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.824549 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7a4ea-04b2-425d-b4e6-0de7d2b5a768 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.829158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.830027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.830343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.830621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] PATH_INFO: `/v3/projects/b1f248a2c3c44f7ebecf0cc04b4f4c71/users/5c1b911bdcf74cd29aee6995bc82f1e5/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.848301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f248a2c3c44f7ebecf0cc04b4f4c71, user_id=5c1b911bdcf74cd29aee6995bc82f1e5, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.851364 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.865734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1415/2824] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:52 2022] PUT /identity/v3/projects/b1f248a2c3c44f7ebecf0cc04b4f4c71/users/5c1b911bdcf74cd29aee6995bc82f1e5/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:52.870335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:52.880930 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeab912-57e9-4af9-af3c-c0e1122dd8eb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:52.959230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3387b-11d5-4dfd-b77c-4d822f15b364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:52.960059 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3387b-11d5-4dfd-b77c-4d822f15b364 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:52.960355 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3387b-11d5-4dfd-b77c-4d822f15b364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:52.960639 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3387b-11d5-4dfd-b77c-4d822f15b364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:52.963224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3387b-11d5-4dfd-b77c-4d822f15b364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:52.965342 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3387b-11d5-4dfd-b77c-4d822f15b364 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:52.981667 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1410/2825] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:52.986460 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:53.000805 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592277b-4215-4c11-b517-ded73d5e79cf admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:53.018781 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:53.020007 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:53.020471 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:53.021461 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] PATH_INFO: `/v3/projects/b1f248a2c3c44f7ebecf0cc04b4f4c71/users/5c1b911bdcf74cd29aee6995bc82f1e5/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:53.025477 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f248a2c3c44f7ebecf0cc04b4f4c71, user_id=5c1b911bdcf74cd29aee6995bc82f1e5, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:53.032545 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:53.048996 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1416/2826] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:52 2022] PUT /identity/v3/projects/b1f248a2c3c44f7ebecf0cc04b4f4c71/users/5c1b911bdcf74cd29aee6995bc82f1e5/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:53.122018 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:53.122388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:53.122679 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:53.155909 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] MFA Rules not processed for user `5c1b911bdcf74cd29aee6995bc82f1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:53.157403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:53.231255 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1411/2827] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:53.242095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:53.245155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:53.308449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d0701-727a-41b2-8f2f-dac5bc16762a admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:53.312158 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84a907-829a-4cc8-8cd5-003b43535c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:53.312809 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a907-829a-4cc8-8cd5-003b43535c3b service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:53.313099 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a907-829a-4cc8-8cd5-003b43535c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:53.313383 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a907-829a-4cc8-8cd5-003b43535c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:53.323435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a907-829a-4cc8-8cd5-003b43535c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:53.324887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a907-829a-4cc8-8cd5-003b43535c3b service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:53.334592 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1417/2828] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:53.728293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:53.741284 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:53.791720 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e0f6e-5731-4a15-ab48-3c02b56a5ac1 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:53.796160 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:53.797034 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:53.797316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:53.797593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] PATH_INFO: `/v3/users/e2686ee37d334f01b8ddd8cefe09b7b0` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:53.798918 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2686ee37d334f01b8ddd8cefe09b7b0)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:53.800867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:53.848595 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2686ee37d334f01b8ddd8cefe09b7b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:53.849422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2686ee37d334f01b8ddd8cefe09b7b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:53.850023 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2686ee37d334f01b8ddd8cefe09b7b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:53.858116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2686ee37d334f01b8ddd8cefe09b7b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:53.872619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2686ee37d334f01b8ddd8cefe09b7b0'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:53.887999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1412/2829] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:53 2022] DELETE /identity/v3/users/e2686ee37d334f01b8ddd8cefe09b7b0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:53.893260 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:53.893573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:53.893936 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:53.912642 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:53.913853 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:53.950375 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:53.982704 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1418/2830] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:53.992398 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:53.994474 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:54.057820 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8765c5-152c-46e9-9e9e-fe5212fda553 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:54.061487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:54.062140 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:54.062443 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:54.062707 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:54.072554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:54.074089 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:54.083539 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1413/2831] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:16:54.392680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:54.401917 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:54.417552 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8ad27-25b8-421e-9514-08dc5c70a2b6 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:54.424720 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:54.425680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:54.425964 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:54.426738 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] PATH_INFO: `/v3/projects/f0632bd8beff43ea817a748a09d6f769` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:54.428709 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0632bd8beff43ea817a748a09d6f769)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:54.432024 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:54.482143 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0632bd8beff43ea817a748a09d6f769', 'request_id': 'req-1f0a7fee-b294-4922-804b-6680da7b9554'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.488326 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0632bd8beff43ea817a748a09d6f769', 'request_id': 'req-1f0a7fee-b294-4922-804b-6680da7b9554'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.496199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0632bd8beff43ea817a748a09d6f769', 'request_id': 'req-1f0a7fee-b294-4922-804b-6680da7b9554'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.506686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] The token cache is being invalidate because project f0632bd8beff43ea817a748a09d6f769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:54.507015 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.507634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.509816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1419/2832] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:54 2022] DELETE /identity/v3/projects/f0632bd8beff43ea817a748a09d6f769 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:54.513785 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:54.524554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5144b-d872-4dee-9288-9fd03fa9a5ac service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:54.560589 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:54.594695 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:54.595602 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:54.595889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:54.596168 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] PATH_INFO: `/v3/users/5c1b911bdcf74cd29aee6995bc82f1e5` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:54.597356 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1b911bdcf74cd29aee6995bc82f1e5)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:54.599656 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:54.646334 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1b911bdcf74cd29aee6995bc82f1e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.646945 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1b911bdcf74cd29aee6995bc82f1e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.647552 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1b911bdcf74cd29aee6995bc82f1e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.655646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1b911bdcf74cd29aee6995bc82f1e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.667689 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-fb5bc4fc-b93f-461f-8a99-720d3798798b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1b911bdcf74cd29aee6995bc82f1e5'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:54.680723 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1414/2833] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:16:54 2022] DELETE /identity/v3/users/5c1b911bdcf74cd29aee6995bc82f1e5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:54.967926 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:54.969493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:54.984253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7fee-b294-4922-804b-6680da7b9554 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.045440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.046310 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.046589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.046890 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] PATH_INFO: `/v3/projects/b1f248a2c3c44f7ebecf0cc04b4f4c71` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.048074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f248a2c3c44f7ebecf0cc04b4f4c71)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.049990 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.096782 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f248a2c3c44f7ebecf0cc04b4f4c71', 'request_id': 'req-f30c0893-fa0f-4f9f-9d14-750156085ae7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:55.104887 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f248a2c3c44f7ebecf0cc04b4f4c71', 'request_id': 'req-f30c0893-fa0f-4f9f-9d14-750156085ae7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:55.112940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f248a2c3c44f7ebecf0cc04b4f4c71', 'request_id': 'req-f30c0893-fa0f-4f9f-9d14-750156085ae7'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:55.115107 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:55.127893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] The token cache is being invalidate because project b1f248a2c3c44f7ebecf0cc04b4f4c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:16:55.128185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:55.128857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f30c0893-fa0f-4f9f-9d14-750156085ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:16:55.130416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1420/2834] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:16:54 2022] DELETE /identity/v3/projects/b1f248a2c3c44f7ebecf0cc04b4f4c71 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:55.137827 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.138323 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.138554 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.159157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:55.160316 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:55.220758 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1415/2835] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:16:55.229734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.230036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.230327 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.249271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:55.250491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:55.258480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1421/2836] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:16:55.263111 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.273390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c06b-f424-48b6-ba19-b5084a02159a None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.279188 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.279880 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.280166 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.280445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.281190 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:16:55.281497 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.283226 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.297795 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1416/2837] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 12:16:55.302449 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.313598 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a0597-6bc2-43a2-bea2-ef085f84558a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.318332 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.319038 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.319328 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.319608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.320299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.322307 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.350362 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1422/2838] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:16:55 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:16:55.355119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.365390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769d566-5220-42c2-95fb-e7671f45b7bb admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.407050 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:16:55.427435 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.428158 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.428506 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.428793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.429487 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.432396 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.452527 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1417/2839] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:16:55 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 12:16:55.456726 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.467345 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee38d-64e8-4190-b2ed-2b944e789ed2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.471904 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.472557 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.472846 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.473126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.473874 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.475798 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.486854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1423/2840] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:55.490982 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.502184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febe20-9fea-4dc7-b2ca-3c0e9bce0bc0 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.506834 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855a8fb-d8e5-438b-adb2-110e781ee4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.507548 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855a8fb-d8e5-438b-adb2-110e781ee4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.507831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855a8fb-d8e5-438b-adb2-110e781ee4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.508122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855a8fb-d8e5-438b-adb2-110e781ee4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.508938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855a8fb-d8e5-438b-adb2-110e781ee4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.510544 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855a8fb-d8e5-438b-adb2-110e781ee4ae admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.520657 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1418/2841] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:16:55.524838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.534857 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fcbee-d444-4fbb-84cd-d3f3e8c13bc5 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.539643 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.541014 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.541360 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.541796 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] PATH_INFO: `/v3/projects/13e993e90963480ea2137b271c4251f8/users/80f0ee31eccc45549eecadd370fbbad0/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.558090 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e993e90963480ea2137b271c4251f8, user_id=80f0ee31eccc45549eecadd370fbbad0, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.560885 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.574634 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1424/2842] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:16:55 2022] PUT /identity/v3/projects/13e993e90963480ea2137b271c4251f8/users/80f0ee31eccc45549eecadd370fbbad0/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:16:55.585908 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16dd66-9f50-47ea-a9b4-2e0eb90ef88c None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.586213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16dd66-9f50-47ea-a9b4-2e0eb90ef88c None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.586505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16dd66-9f50-47ea-a9b4-2e0eb90ef88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.621072 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-cc16dd66-9f50-47ea-a9b4-2e0eb90ef88c None None] MFA Rules not processed for user `80f0ee31eccc45549eecadd370fbbad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:16:55.622422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-cc16dd66-9f50-47ea-a9b4-2e0eb90ef88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:55.693262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1419/2843] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:16:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:16:55.704211 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:16:55.705943 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:16:55.763271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910db95-fabb-4779-a2c1-33247d6189e2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:16:55.766893 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:16:55.767540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:16:55.767836 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:16:55.768114 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:16:55.777394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:16:55.778859 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:16:55.787907 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1425/2844] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:05.690219 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:05.784167 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=56,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:06.190411 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d223d-fc26-45d3-b153-b2685cb453aa None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:06.190706 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d223d-fc26-45d3-b153-b2685cb453aa None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:06.191016 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d223d-fc26-45d3-b153-b2685cb453aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:06.209637 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-286d223d-fc26-45d3-b153-b2685cb453aa None None] MFA Rules not processed for user `8c4a7545593a45dca9c0029d29910811`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:17:06.210938 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-286d223d-fc26-45d3-b153-b2685cb453aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:06.268979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1420/2845] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:17:06.625291 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:06.707205 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc499-4e83-4a0b-b089-32a5b5bb6f2a service cinder] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:06.712875 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:06.713883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:06.714297 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:06.714671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] PATH_INFO: `/v3/users/80f0ee31eccc45549eecadd370fbbad0` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:06.717529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f0ee31eccc45549eecadd370fbbad0)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:06.719665 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:06.785950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f0ee31eccc45549eecadd370fbbad0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:06.786644 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f0ee31eccc45549eecadd370fbbad0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:06.796680 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f0ee31eccc45549eecadd370fbbad0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:06.797410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f0ee31eccc45549eecadd370fbbad0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:06.806489 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f0ee31eccc45549eecadd370fbbad0'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:06.827837 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1426/2846] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:17:06 2022] DELETE /identity/v3/users/80f0ee31eccc45549eecadd370fbbad0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:17:06.833176 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:06.834001 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:06.834570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:06.859126 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:17:06.860390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:06.948584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1421/2847] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:17:06.962900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:06.964949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:07.049827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb215e-b8e9-47c7-aa25-b2d15893fc00 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:07.054703 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22ed1c-dba2-43fc-ae74-3212725aadc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:07.055487 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22ed1c-dba2-43fc-ae74-3212725aadc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.055896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22ed1c-dba2-43fc-ae74-3212725aadc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.056250 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22ed1c-dba2-43fc-ae74-3212725aadc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.070661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22ed1c-dba2-43fc-ae74-3212725aadc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:07.072173 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22ed1c-dba2-43fc-ae74-3212725aadc5 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:07.084854 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1427/2848] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:07.431781 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:07.433359 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:07.448889 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516db966-fffc-4d34-85c9-9c330eede9b6 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:07.456538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:07.457326 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.457642 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.458455 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] PATH_INFO: `/v3/projects/13e993e90963480ea2137b271c4251f8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.459449 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e993e90963480ea2137b271c4251f8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:07.461630 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:07.527317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e993e90963480ea2137b271c4251f8', 'request_id': 'req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:07.535218 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e993e90963480ea2137b271c4251f8', 'request_id': 'req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:07.543964 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e993e90963480ea2137b271c4251f8', 'request_id': 'req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:07.558486 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] The token cache is being invalidate because project 13e993e90963480ea2137b271c4251f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:17:07.559559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:07.560551 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-f63b142f-6f8a-418a-9d5d-426cf7e3eab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:07.562189 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1422/2849] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:17:07 2022] DELETE /identity/v3/projects/13e993e90963480ea2137b271c4251f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:17:07.567216 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.567653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.568079 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.597343 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:17:07.599190 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:07.625833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:07.687430 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1428/2850] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:17:07.696043 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.696351 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.696635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.716377 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:17:07.717694 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:07.726194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1423/2851] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:17:07.730472 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:07.740900 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987457ee-d3db-4644-8887-a857cb7777ce None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:07.745657 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:07.746955 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.747723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.747723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.748354 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:17:07.748668 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:07.750301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:07.764301 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1429/2852] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:17:07.769268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:07.779281 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c9c9a-00d1-4bab-9959-6e85207ac5a2 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:07.784086 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:07.784804 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.785138 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.785462 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.786203 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:07.788560 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:07.813696 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:07.826388 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1424/2853] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:17:07 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:17:07.827868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:07.838773 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd9b6e-faf1-4819-a234-371b44fbf959 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:07.913259 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:07.913949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.914232 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.914550 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.915263 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:07.917305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:07.940228 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1430/2854] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:17:07 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:17:07.944561 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:07.961703 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9af87-7664-458c-bb58-2b69b50104f1 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:07.967740 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:07.968440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:07.968712 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:07.969037 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:07.969941 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:07.971621 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:07.985175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1425/2855] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:17:07.989840 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:08.008424 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea9af-abc0-4c85-80a7-57aa472309ba admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:08.018940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8169101-80ab-4c45-bb0e-5673eaee454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:08.019198 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8169101-80ab-4c45-bb0e-5673eaee454e admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:08.019591 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8169101-80ab-4c45-bb0e-5673eaee454e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:08.020089 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8169101-80ab-4c45-bb0e-5673eaee454e admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:08.020968 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8169101-80ab-4c45-bb0e-5673eaee454e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:08.024182 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8169101-80ab-4c45-bb0e-5673eaee454e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:08.039722 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1431/2856] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:17:08.044593 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:08.058681 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868c4b2-7544-4d1c-b926-524ae588b347 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:08.073986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:08.076644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:08.077046 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:08.079420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] PATH_INFO: `/v3/projects/51302524c5e846899cad4eca5d0d20f7/users/2a0d1e709c2f4a1484a9d7e19d4111a8/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:08.103680 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51302524c5e846899cad4eca5d0d20f7, user_id=2a0d1e709c2f4a1484a9d7e19d4111a8, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:08.106910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:08.134204 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1426/2857] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:17:08 2022] PUT /identity/v3/projects/51302524c5e846899cad4eca5d0d20f7/users/2a0d1e709c2f4a1484a9d7e19d4111a8/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:17:08.145869 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:08.146172 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:08.146463 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:08.188043 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] MFA Rules not processed for user `2a0d1e709c2f4a1484a9d7e19d4111a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:17:08.189413 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:08.276299 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1432/2858] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:17:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:17:08.291260 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:08.294532 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:08.380922 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74caba8-409b-4c53-94bc-ab52f8c41cb9 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:08.384715 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '106467a9f0644177872555b7998599d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:08.385382 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:08.385674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:08.385956 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:08.395747 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:08.396926 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:08.407426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1427/2859] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:15.126199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:15.127716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:15.189508 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:15.371253 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ef49-8f7e-4705-9d21-3a626e827312 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:15.381363 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:15.382034 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:15.382323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:15.382604 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:15.385140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:15.407414 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:15.410142 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936c9a0-cb8d-4840-bfd6-a28aa1a0cd56 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:15.427721 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1433/2860] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:18.402105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:19.086098 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:19.174387 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd354be-cb09-4a34-b91d-ce376f810b0f service cinder] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:19.179671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:19.180516 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:19.180801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:19.181108 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] PATH_INFO: `/v3/users/2a0d1e709c2f4a1484a9d7e19d4111a8` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:19.182542 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a0d1e709c2f4a1484a9d7e19d4111a8)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:19.184750 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:19.249633 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a0d1e709c2f4a1484a9d7e19d4111a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.251066 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a0d1e709c2f4a1484a9d7e19d4111a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.252445 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a0d1e709c2f4a1484a9d7e19d4111a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.266528 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a0d1e709c2f4a1484a9d7e19d4111a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.287513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a0d1e709c2f4a1484a9d7e19d4111a8'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.302748 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1428/2861] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:17:19 2022] DELETE /identity/v3/users/2a0d1e709c2f4a1484a9d7e19d4111a8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:17:19.307410 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:19.307937 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:19.308036 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:19.334653 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:17:19.335940 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:19.402074 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1434/2862] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:17:19.412605 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:19.414384 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:19.473972 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d7eb9-8f02-4ccf-81e5-ea28b7a33519 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:19.477739 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:19.478645 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:19.478645 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:19.478979 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:19.489019 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:19.490454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:19.500456 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1429/2863] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:19.811539 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:19.813393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:19.827334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895ba62-ca58-43ee-8b59-57e7a29d400a None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:19.832206 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:19.833126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:19.833417 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:19.833696 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] PATH_INFO: `/v3/projects/51302524c5e846899cad4eca5d0d20f7` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:19.834946 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51302524c5e846899cad4eca5d0d20f7)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:19.836949 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:19.890740 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51302524c5e846899cad4eca5d0d20f7', 'request_id': 'req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.898555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51302524c5e846899cad4eca5d0d20f7', 'request_id': 'req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.912916 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51302524c5e846899cad4eca5d0d20f7', 'request_id': 'req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.924199 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:19.929870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] The token cache is being invalidate because project 51302524c5e846899cad4eca5d0d20f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:17:19.929870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.930486 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:19.932527 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1435/2864] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:17:19 2022] DELETE /identity/v3/projects/51302524c5e846899cad4eca5d0d20f7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:17:28.297727 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:28.299349 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:28.311711 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8d2c3-b246-4166-92ef-098f2974a17c service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:28.331987 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:28.375867 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:28.376726 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:28.377141 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:28.377535 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:28.379800 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:28.475573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:28.475573 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:28.479063 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1430/2865] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 12:17:31.784318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:31.785958 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:31.899447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e9d1-e32c-4f36-b704-44cd34d86dd8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:31.905275 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:31.905933 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:31.906221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:31.906506 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:31.908366 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:31.920470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:31.922095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:31.932529 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1436/2866] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:35.135505 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:35.137427 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:35.151391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f11200-5285-47a2-8c4b-8640677f5de7 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:35.156891 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:35.157828 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:35.158119 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:35.158393 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] PATH_INFO: `/v3/users/adb92379209e46e79de4df370183fa8a` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:35.172817 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb92379209e46e79de4df370183fa8a)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:35.174805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:35.220884 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb92379209e46e79de4df370183fa8a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.221546 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb92379209e46e79de4df370183fa8a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.222164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb92379209e46e79de4df370183fa8a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.229584 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb92379209e46e79de4df370183fa8a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.241313 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb92379209e46e79de4df370183fa8a'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.254415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1431/2867] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:17:35 2022] DELETE /identity/v3/users/adb92379209e46e79de4df370183fa8a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:17:35.440403 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:35.449847 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af5705-9bf9-4151-8841-d89d2bfb8c86 service placement] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:35.511221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:35.511998 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:35.512285 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:35.512567 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] PATH_INFO: `/v3/projects/c80560ca253e41239020e2c008a9b3ba` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:35.522860 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80560ca253e41239020e2c008a9b3ba)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:35.524790 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:35.570126 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80560ca253e41239020e2c008a9b3ba', 'request_id': 'req-a570c3da-8225-4c82-8c59-eadb770e2ab8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.577060 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80560ca253e41239020e2c008a9b3ba', 'request_id': 'req-a570c3da-8225-4c82-8c59-eadb770e2ab8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.584409 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80560ca253e41239020e2c008a9b3ba', 'request_id': 'req-a570c3da-8225-4c82-8c59-eadb770e2ab8'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.586651 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:35.598272 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] The token cache is being invalidate because project c80560ca253e41239020e2c008a9b3ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:17:35.598565 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.599225 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.601086 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1437/2868] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:17:35 2022] DELETE /identity/v3/projects/c80560ca253e41239020e2c008a9b3ba => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:17:35.605184 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:35.606570 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:35.620311 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ee2be-3391-44fd-9d61-da4135dfd0cc admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:35.650068 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:35.679082 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:35.679962 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:35.680252 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:35.680538 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] PATH_INFO: `/v3/users/7d6cea527f36408989e27a6ca577371e` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:35.694137 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6cea527f36408989e27a6ca577371e)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:35.696116 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:35.745105 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6cea527f36408989e27a6ca577371e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.745988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6cea527f36408989e27a6ca577371e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.746619 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6cea527f36408989e27a6ca577371e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.755014 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6cea527f36408989e27a6ca577371e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.768262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6cea527f36408989e27a6ca577371e'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:35.781744 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1432/2869] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:17:35 2022] DELETE /identity/v3/users/7d6cea527f36408989e27a6ca577371e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:17:35.970305 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:35.971764 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:35.990973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570c3da-8225-4c82-8c59-eadb770e2ab8 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:36.048589 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:36.049464 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:36.049752 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:36.050032 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] PATH_INFO: `/v3/projects/c2be2e7a09154df2a1c2b1bbc17c1094` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:36.061822 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2be2e7a09154df2a1c2b1bbc17c1094)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:36.064028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:36.117855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2be2e7a09154df2a1c2b1bbc17c1094', 'request_id': 'req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:36.124994 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2be2e7a09154df2a1c2b1bbc17c1094', 'request_id': 'req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:36.132666 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2be2e7a09154df2a1c2b1bbc17c1094', 'request_id': 'req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:36.143867 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] The token cache is being invalidate because project c2be2e7a09154df2a1c2b1bbc17c1094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80147) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:17:36.144255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:36.144810 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:17:36.146384 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1438/2870] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:17:35 2022] DELETE /identity/v3/projects/c2be2e7a09154df2a1c2b1bbc17c1094 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:17:45.777296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:46.140701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=58,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:17:46.738499 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:17:46.739977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:46.811769 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86759b-f107-4de8-9eee-55dccfa8de02 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:17:46.872144 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:17:46.872805 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:17:46.873095 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:17:46.873380 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:17:46.875209 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:17:46.888697 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:17:46.890335 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:17:46.901644 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1433/2871] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:17:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:17:56.896671 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=58 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:12.313945 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:12.315648 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:12.397020 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e7e98-3814-4a55-9bdc-93398cf9e8aa admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:12.402280 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:12.403235 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:12.403548 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:12.403856 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] PATH_INFO: `/v3/users/8c4a7545593a45dca9c0029d29910811` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:12.405279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4a7545593a45dca9c0029d29910811)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:12.407480 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:12.467838 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4a7545593a45dca9c0029d29910811'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.468482 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4a7545593a45dca9c0029d29910811'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.476676 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4a7545593a45dca9c0029d29910811'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.477814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4a7545593a45dca9c0029d29910811'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.487006 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-c07bc0d3-f5cd-4654-b104-4ca0714395ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4a7545593a45dca9c0029d29910811'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.507016 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1439/2872] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:18:12 2022] DELETE /identity/v3/users/8c4a7545593a45dca9c0029d29910811 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:18:12.709202 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:12.720731 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f889c6-402b-464a-9ac7-acf745f00e0a service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:12.795412 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:12.796293 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:12.796586 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:12.796868 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] PATH_INFO: `/v3/projects/621a97bcf68f4dea8bce2be900c2eefd` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:12.798083 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=621a97bcf68f4dea8bce2be900c2eefd)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:12.800246 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:12.848977 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621a97bcf68f4dea8bce2be900c2eefd', 'request_id': 'req-a49139fa-76a0-410c-bac2-7c3779c0d7c9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.856013 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621a97bcf68f4dea8bce2be900c2eefd', 'request_id': 'req-a49139fa-76a0-410c-bac2-7c3779c0d7c9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.863117 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621a97bcf68f4dea8bce2be900c2eefd', 'request_id': 'req-a49139fa-76a0-410c-bac2-7c3779c0d7c9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.878465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] The token cache is being invalidate because project 621a97bcf68f4dea8bce2be900c2eefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:18:12.878797 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.879722 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-a49139fa-76a0-410c-bac2-7c3779c0d7c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:18:12.881914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1434/2873] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:18:12 2022] DELETE /identity/v3/projects/621a97bcf68f4dea8bce2be900c2eefd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 12:18:12.888716 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:12.889192 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:12.889621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:12.911787 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:12.913239 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:12.994108 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1440/2874] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 12:18:13.002337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.002652 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.003100 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.025374 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:13.026789 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:13.035015 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1435/2875] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:13.041499 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.052129 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6be7-971f-4b40-945c-b1d1c5768d9c None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.066185 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.067050 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.067415 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.067671 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] PATH_INFO: `/v3/domains` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.068425 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80147) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 12:18:13.068737 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.070467 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.084573 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1441/2876] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 12:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 12:18:13.089582 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.099713 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d1671-0f57-490f-adf4-9599b69ba9b0 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.104674 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.105324 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.105635 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.105914 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.106609 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.108585 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.136420 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1436/2877] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:18:13 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:18:13.140758 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.151872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2ed0b-d425-4746-b24f-dee4aa815853 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.155645 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:13.225536 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.226217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.226540 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.226827 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.227852 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.229814 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.258071 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1442/2878] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:18:13 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:18:13.262213 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.274873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14174e-2c4f-4ab8-b8dc-9f5b3b4d051d admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.279831 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.280507 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.280793 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.281147 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.281810 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.283618 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.296811 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1437/2879] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:18:13.302791 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.312831 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d98db-5a2e-4297-bc2d-c5f77eea8f38 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.320174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1937b17-e8ad-4fec-9c6e-b05a0b7d3b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.320174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1937b17-e8ad-4fec-9c6e-b05a0b7d3b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.320174 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1937b17-e8ad-4fec-9c6e-b05a0b7d3b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.321528 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1937b17-e8ad-4fec-9c6e-b05a0b7d3b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.321725 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1937b17-e8ad-4fec-9c6e-b05a0b7d3b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.323594 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1937b17-e8ad-4fec-9c6e-b05a0b7d3b59 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.335304 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1443/2880] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 12:18:13.340391 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.354157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4f600-583b-4dae-9fed-7c4793b7394a admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.359999 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.360888 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.361175 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.361459 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] PATH_INFO: `/v3/projects/101a824926d14b078830ace554e9ff3d/users/7c132e34c7e341f9bd753efd38f4fbbd/roles/5a7ddb9a26de4c15abc5d71777d9e61b` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.377646 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=101a824926d14b078830ace554e9ff3d, user_id=7c132e34c7e341f9bd753efd38f4fbbd, role_id=5a7ddb9a26de4c15abc5d71777d9e61b)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.381346 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.397306 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1438/2881] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:18:13 2022] PUT /identity/v3/projects/101a824926d14b078830ace554e9ff3d/users/7c132e34c7e341f9bd753efd38f4fbbd/roles/5a7ddb9a26de4c15abc5d71777d9e61b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:18:13.403027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190ccff-967d-4ad8-bb05-e70b9accd271 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.403329 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190ccff-967d-4ad8-bb05-e70b9accd271 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.403612 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190ccff-967d-4ad8-bb05-e70b9accd271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.424855 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-e190ccff-967d-4ad8-bb05-e70b9accd271 None None] MFA Rules not processed for user `b408ca5ee3cd4f8b9e1f61c55e26955b`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:13.427896 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e190ccff-967d-4ad8-bb05-e70b9accd271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:13.515611 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1444/2882] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:13.527017 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:13.528919 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:13.563224 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:13.598912 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931143ee-9a97-42ed-90f4-9d3a87168dac admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:13.603156 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:13.603876 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:13.604167 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:13.604454 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:13.613906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:13.615441 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:13.625247 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1439/2883] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:18:17.985914 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:17.986575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:17.986575 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.017656 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] MFA Rules not processed for user `7c132e34c7e341f9bd753efd38f4fbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:18.018939 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:18.111044 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1445/2884] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:18:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:18.119913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:18.128955 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374eaf-70dd-4f5c-b709-ce8385a44a8b service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:18.137426 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:18.138103 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:18.138390 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:18.138670 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.139410 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:18.141440 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:18.182408 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1440/2885] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 12:18:18 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:18:18.187623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:18.202221 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1359d6-5543-4730-8ba6-ad313b94834d None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:18.223179 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:18.313650 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:18.316095 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:18.318296 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:18.318807 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.319950 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:18.322271 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:18.345309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1446/2886] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 12:18:18 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 12:18:18.350261 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:18.368837 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b388d6-7274-4fa3-bb6a-5489f6a3f2ce admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:18.374545 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:18.375041 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:18.375415 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:18.375592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.376378 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:18.378262 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:18.394056 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1441/2887] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:18:18.399889 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:18.415678 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a9e5f-2f70-4e03-a82a-e3cb97ec74c2 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:18.420872 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:18.421833 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:18.422140 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:18.422427 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] PATH_INFO: `/v3/projects/3b403c15cd3042aead13fdafedb893a3/users/440342e4329c4938bd306363199655dc/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.443442 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b403c15cd3042aead13fdafedb893a3, user_id=440342e4329c4938bd306363199655dc, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:18.447317 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:18.466396 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1447/2888] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:18:18 2022] PUT /identity/v3/projects/3b403c15cd3042aead13fdafedb893a3/users/440342e4329c4938bd306363199655dc/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:18:18.470264 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:18.484825 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a770e0-6c14-4241-b247-4a535bae77b6 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:18.567058 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e2565-42ac-4b72-9d70-9bbdbd48bcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:18.568466 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e2565-42ac-4b72-9d70-9bbdbd48bcd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:18.569274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e2565-42ac-4b72-9d70-9bbdbd48bcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:18.569274 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e2565-42ac-4b72-9d70-9bbdbd48bcd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.570404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e2565-42ac-4b72-9d70-9bbdbd48bcd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:18.572431 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e2565-42ac-4b72-9d70-9bbdbd48bcd9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:18.590478 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:18.592913 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1442/2889] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 12:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 12:18:18.603723 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07984e01-5cca-42d9-a01d-439fbd896fd7 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:18.610258 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:18.613491 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] REQUEST_METHOD: `PUT` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:18.614815 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:18.616072 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] PATH_INFO: `/v3/projects/3b403c15cd3042aead13fdafedb893a3/users/440342e4329c4938bd306363199655dc/roles/b4418dc4009641b183fe42e56f1e484e` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:18.622393 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b403c15cd3042aead13fdafedb893a3, user_id=440342e4329c4938bd306363199655dc, role_id=b4418dc4009641b183fe42e56f1e484e)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:18.625734 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:18.650155 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1448/2890] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 12:18:18 2022] PUT /identity/v3/projects/3b403c15cd3042aead13fdafedb893a3/users/440342e4329c4938bd306363199655dc/roles/b4418dc4009641b183fe42e56f1e484e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:18:24.275784 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68a34b-52e2-4c27-8274-91ec2b134c13 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:24.276367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68a34b-52e2-4c27-8274-91ec2b134c13 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:24.276367 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68a34b-52e2-4c27-8274-91ec2b134c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:24.308895 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-ea68a34b-52e2-4c27-8274-91ec2b134c13 None None] MFA Rules not processed for user `440342e4329c4938bd306363199655dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:24.310164 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-ea68a34b-52e2-4c27-8274-91ec2b134c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:24.415737 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1443/2891] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:18:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:24.432390 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:24.435027 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:24.552444 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd7118-9272-4d23-b105-9b22352da09f admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:24.557635 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0fa7ced305e4d2fa0d791675f839453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:24.558323 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:24.558616 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:24.558930 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:24.641524 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:24.644713 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:24.660255 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1449/2892] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:18:24.814304 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a421-73b2-4fc8-8d50-1edc0615fb96 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:24.814599 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a421-73b2-4fc8-8d50-1edc0615fb96 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:24.815142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a421-73b2-4fc8-8d50-1edc0615fb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:24.846894 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:24.892298 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-d534a421-73b2-4fc8-8d50-1edc0615fb96 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:24.893547 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-d534a421-73b2-4fc8-8d50-1edc0615fb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:24.979312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1444/2893] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:18:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:25.031862 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:25.033292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:25.039303 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:25.137748 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e0bf6-27ab-42a8-94d3-549a5b6ee59d service glance] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:25.144308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3764067-e4f9-4a4d-b4e0-6969c1fa4a00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:25.144973 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764067-e4f9-4a4d-b4e0-6969c1fa4a00 service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:25.145804 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764067-e4f9-4a4d-b4e0-6969c1fa4a00 service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:25.146203 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764067-e4f9-4a4d-b4e0-6969c1fa4a00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:25.156154 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3764067-e4f9-4a4d-b4e0-6969c1fa4a00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:25.157818 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3764067-e4f9-4a4d-b4e0-6969c1fa4a00 service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:25.168214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1450/2894] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:18:34.028978 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:34.029471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:34.029571 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:34.049341 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] MFA Rules not processed for user `7c132e34c7e341f9bd753efd38f4fbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:34.050559 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:34.058139 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1445/2895] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 12:18:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:35.168585 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=40 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:35.897394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] REQUEST_METHOD: `POST` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:35.897695 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:35.897983 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:35.916447 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.auth.core [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80147) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:18:35.917647 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:35.925061 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1451/2896] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 12:18:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:18:35.937266 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:35.951232 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d17d1-6330-41e3-ba35-733cfe301d09 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:35.956453 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:35.957207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:35.957500 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:35.958376 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:35.970873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:35.970873 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:35.980102 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1446/2897] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:18:45.321147 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:18:45.379600 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02491304-e2bf-4125-9301-e56c1f100482 None None] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:18:45.383334 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:18:45.383992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:18:45.384279 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:18:45.384562 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:18:45.387073 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:18:45.462395 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:18:45.463883 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:18:45.476440 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1452/2898] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:18:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:18:45.976296 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=45 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:18:55.474751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=57 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:19:06.447910 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:19:06.449404 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:19:06.463419 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956084-4edb-498f-a144-de09a429f798 service swift] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:19:06.467044 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:19:06.467699 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:19:06.467988 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:19:06.468272 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:19:06.478005 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:19:06.479513 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:19:06.490517 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1447/2899] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:19:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:19:08.687701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:19:08.699049 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbaa38-2be5-428b-96cb-74b05f434652 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:19:08.704621 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:19:08.705470 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:19:08.705868 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:19:08.706268 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:19:08.709102 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:19:08.721776 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:19:08.723370 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:19:08.733405 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1453/2900] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:19:16.485483 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:19:18.729214 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=6 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:20:09.927317 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:20:09.938157 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cce7a-999f-4475-9cc7-780e95d8d0c0 service neutron] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:20:09.943815 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2665b33ffa38466a98a5e4ea838916c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:20:09.944479 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:20:09.944838 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:20:09.945153 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:20:09.947312 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:20:09.965438 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:20:09.967709 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a96d91-bf9d-49ab-83a4-c106abf395cf service neutron] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:20:09.980905 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1448/2901] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:20:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:20:19.974480 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=6 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:21:31.052344 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:31.053992 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:31.071684 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e20cf-f4e2-417b-9f3d-dbba60ff5f72 service neutron] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:31.078406 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0208c5889a74870b764c7eb954242bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:31.079123 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:31.079816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:31.079816 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:31.081493 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:31.095068 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:31.096701 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:31.108686 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1454/2902] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:21:32.198685 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] REQUEST_METHOD: `POST` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:32.199207 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:32.199319 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:32.217092 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.auth.core [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] MFA Rules not processed for user `5b62f565650b42f5b4a1bc2f8b39b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 12:21:32.218301 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:32.225439 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1449/2903] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 12:21:32.267757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:32.279438 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df854751-b0b0-4749-aeb4-5145de956182 service nova] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:32.284555 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a914be52bf745d4b4647f482460e734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:32.285226 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:32.285516 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:32.285800 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:32.294757 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:32.296380 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:32.304475 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1455/2904] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 12:21:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 12:21:35.802614 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:35.804194 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:35.896069 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb193-90ce-4e97-82c2-7b7a767b8282 None None] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:35.902142 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:35.902813 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] REQUEST_METHOD: `GET` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:35.903212 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:35.903592 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:35.905526 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:35.921465 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:35.923122 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:35.934986 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1450/2905] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:21:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:21:39.048217 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:39.119265 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a486c3-68ea-469c-bdf8-29a252ba859d service swift] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:39.125004 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:39.125870 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:39.126156 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:39.126435 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] PATH_INFO: `/v3/users/7c132e34c7e341f9bd753efd38f4fbbd` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:39.127652 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c132e34c7e341f9bd753efd38f4fbbd)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:39.129608 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:39.178309 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c132e34c7e341f9bd753efd38f4fbbd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.179025 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c132e34c7e341f9bd753efd38f4fbbd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.186624 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c132e34c7e341f9bd753efd38f4fbbd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.193292 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c132e34c7e341f9bd753efd38f4fbbd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.195028 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c132e34c7e341f9bd753efd38f4fbbd'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.214256 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1456/2906] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:21:39 2022] DELETE /identity/v3/users/7c132e34c7e341f9bd753efd38f4fbbd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:21:39.407268 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:39.418536 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ba93f-41db-450f-adde-d38942b8fa3e service placement] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:39.478683 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:39.479617 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:39.479906 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:39.480177 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] PATH_INFO: `/v3/projects/101a824926d14b078830ace554e9ff3d` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:39.481337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=101a824926d14b078830ace554e9ff3d)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:39.483288 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:39.529429 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101a824926d14b078830ace554e9ff3d', 'request_id': 'req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.535079 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101a824926d14b078830ace554e9ff3d', 'request_id': 'req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.541422 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101a824926d14b078830ace554e9ff3d', 'request_id': 'req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.555125 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] The token cache is being invalidate because project 101a824926d14b078830ace554e9ff3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:21:39.555482 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.556062 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.558403 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1451/2907] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:21:39 2022] DELETE /identity/v3/projects/101a824926d14b078830ace554e9ff3d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:21:39.562439 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:39.563905 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:39.577788 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6f5e-3ef3-4f00-b1df-3567fdee3729 admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:39.628201 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:21:39.638429 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:39.639374 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:39.639661 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:39.639941 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] PATH_INFO: `/v3/users/440342e4329c4938bd306363199655dc` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:39.641119 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] RBAC: Authorizing `identity:delete_user(user_id=440342e4329c4938bd306363199655dc)` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:39.643141 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:39.689622 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440342e4329c4938bd306363199655dc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.690294 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440342e4329c4938bd306363199655dc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.700394 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440342e4329c4938bd306363199655dc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.701416 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440342e4329c4938bd306363199655dc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.710311 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.notifications [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440342e4329c4938bd306363199655dc'} {{(pid=80147) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:39.728559 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1457/2908] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 12:21:39 2022] DELETE /identity/v3/users/440342e4329c4938bd306363199655dc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:21:39.907471 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Authenticating user token {{(pid=80146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:21:39.908881 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.fernet_utils [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:21:39.922627 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f782cf5-6b2a-4c9e-b2bc-a7c4d0a22cb2 admin admin] Validating token access rules against request {{(pid=80146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:21:39.932010 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:21:39.984993 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b408ca5ee3cd4f8b9e1f61c55e26955b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a70232cae3264ff0a2f3b7ce99f87508', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:21:39.985775 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:21:39.986061 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:21:39.986337 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] PATH_INFO: `/v3/projects/3b403c15cd3042aead13fdafedb893a3` {{(pid=80146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:21:39.987801 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b403c15cd3042aead13fdafedb893a3)` {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:21:39.989753 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] RBAC: Authorization granted {{(pid=80146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:21:40.038171 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b403c15cd3042aead13fdafedb893a3', 'request_id': 'req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:40.044315 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b403c15cd3042aead13fdafedb893a3', 'request_id': 'req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:40.050417 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b403c15cd3042aead13fdafedb893a3', 'request_id': 'req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9'} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:40.062935 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] The token cache is being invalidate because project 3b403c15cd3042aead13fdafedb893a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 12:21:40.063230 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:40.063846 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG keystone.notifications [None req-ed3d8f8e-ced7-4574-95e1-e0dca06ecff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 12:21:40.065373 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: [pid: 80146|app: 0|req: 1452/2909] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 12:21:39 2022] DELETE /identity/v3/projects/3b403c15cd3042aead13fdafedb893a3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 12:21:49.724751 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:21:50.059886 dv-cinder-825492-20-iscsi devstack@keystone.service[80146]: DEBUG dbcounter [-] [80146] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 12:22:56.668490 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Authenticating user token {{(pid=80147) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 12:22:56.669923 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:22:56.742408 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ea7c0-0636-498e-b10a-d563c2e5a01e admin admin] Validating token access rules against request {{(pid=80147) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 12:22:56.805059 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c11a2d80dcf4533a2b87fb5d85a552c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf85d790611946309fcb27d7ad7aef17', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80147) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 12:22:56.805745 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] REQUEST_METHOD: `GET` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 12:22:56.806031 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] SCRIPT_NAME: `/identity` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 12:22:56.806308 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80147) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 12:22:56.808318 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.fernet_utils [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80147) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 12:22:56.822075 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 12:22:56.823728 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d0e6-663b-4a20-8be7-a7d1c4b9ac84 service placement] RBAC: Authorization granted {{(pid=80147) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 12:22:56.834623 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: [pid: 80147|app: 0|req: 1458/2910] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 12:23:06.830607 dv-cinder-825492-20-iscsi devstack@keystone.service[80147]: DEBUG dbcounter [-] [80147] Writing DB stats keystone:SELECT=58 {{(pid=80147) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}